KR100913092B1 - Method for displaying user interface of media signal, and apparatus for implementing the same - Google Patents

Abstract

The present invention relates to an interface display method and apparatus for displaying a mixed signal that can provide an interface to a user with respect to a mixed signal such as an audio signal and a video signal, the method comprising: selecting a specific mixed signal from a user; And querying the source signal information corresponding to the mixed signal and displaying an icon corresponding to the source signal based on the source signal information.

According to the present invention, information about a source signal (or an object signal) associated with a mixed signal (or a composite object signal) can be provided through a Graphic User Interface (GUI), and an audio signal and a video signal While a mixed signal (or a composite object signal) such as the like is being reproduced, an image representing a source signal associated with the mixed signal (or a composite object signal) may be variously expressed at various times.

Mix, video, audio, object

Description

METHOD FOR DISPLAYING USER INTERFACE OF MEDIA SIGNAL, AND APPARATUS FOR IMPLEMENTING THE SAME}

1 is a block diagram of an apparatus for encoding a first remix signal according to an embodiment of the present invention.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when using a stereo signal. FIG.

3 is a domain for processing a mix signal according to an embodiment of the present invention.

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention.

FIG. 6 is a detailed view of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. FIG.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention.

8B is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention used in combination with a conventional decoding apparatus.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention.

10 is a block diagram of an interface display device according to an embodiment of the present invention.

11A to 11D illustrate examples of an interface display device according to an embodiment of the present invention.

12 is a block diagram of an interface display device according to another embodiment of the present invention.

13A and 13B are flowcharts of an interface display method according to an embodiment of the present invention.

14A is an example of a screen displaying an icon for each source signal of a specific mix signal.

14B is an example of a screen in which an icon for each source signal of a specific mix signal expresses metadata.

15A is an example of a level adjustment screen.

15B is an example of the screen in which the shape of the icon is changed according to the adjusted level.

16 shows an example of an equalizer screen (subband level adjustment screen).

17 is an example of an ambience adjustment screen (relative position adjustment screen).

18 is a block diagram of an interface display device according to another embodiment of the present invention.

19 is a block diagram of an interface display device according to another embodiment of the present invention.

20 is a flowchart illustrating an interface display method according to another embodiment of the present invention.

21 is an example of a screen in which an image size changes in proportion to a level of a source signal.

22 is an example of a screen in which the brightness of an image changes in proportion to the level of a source signal.

23 and 24 are examples of screens in which only an image having the current largest level among source signals is displayed.

The present invention relates to a method and apparatus for displaying an interface of a mixed signal, and more particularly, to an interface displaying method and apparatus for providing an interface to a user with respect to a mixed signal such as an audio signal and a video signal.

In general, stereo signals are most commonly generated and most widely used by consumers. In recent years, multichannel signals have been increasingly used. However, there is a limitation that the mixed signal is processed not in the unit of the source signal constituting the mixed signal but in the channel signal unit. Therefore, when the mixed signal is processed in units of channel signals, only a specific source signal constituting the mixed signal cannot be processed independently. For example, while watching a movie, it is impossible to increase the volume of background music while keeping the volume of actors' voices constant.

In addition, in order for a user to remix a mixed signal (or a composite object signal) by source, that is, by source signal, the user needs to know what source signals are included in the mixed signal. For example, if the mix signal is a music file, the user must know if the source signals associated with the music file are piano, violin, flute, or the like. However, in the related art, only the information of the mixed signal (or the composite object signal) may be displayed, but there is a problem in which the information of the source signals related to the mixed signal is not displayed.

The present invention was devised to solve the above problems, and provides information about a source signal (or an object signal) related to a mixed signal (or a composite object signal) through a Graphic User Interface (GUI). An object of the present invention is to provide an interface display method and apparatus for providing a mixed signal.

It is still another object of the present invention to provide a time varying image representing a source signal associated with a mixed signal (or a composite object signal) while a mixed signal (or a composite object signal) such as an audio signal or a video signal is being reproduced. It is an object of the present invention to provide a method and apparatus for displaying an interface of a mixed signal that can be expressed by changing.

In order to achieve the above object, an interface display method of a mixed signal according to the present invention includes: selecting a specific mixed signal from a user; And querying source signal information corresponding to the mixed signal and displaying an icon corresponding to the source signal based on the source signal information.

According to the present invention, the source signal information may be extracted from additional information corresponding to the mix signal.

According to the present invention, the method includes extracting metadata corresponding to the mixed signal; The icon may be an image matching the metadata.

According to the present invention, the metadata may be extracted from additional information corresponding to the mixed signal, and may include one or more of a musical instrument performer name and a singer name of the source signal.

According to the present invention, if an execution command for one of the icons is received, the method may further include displaying a level adjustment screen of a source signal corresponding to the selected icon.

According to the present invention, the method further includes the step of changing the shape of the selected icon according to the adjusted level when a command regarding level adjustment is received, wherein the shape of the icon includes one or more of size, color, and brightness. It may be.

According to the present invention, if a command for level adjustment is received, the method may further include adjusting a level of a source signal corresponding to the selected icon to remix the mixed signal.

According to the present invention, if one of the icons is selected, the method may further include displaying an equalizer screen of the source signal corresponding to the selected icon.

According to the present invention, when the command for adjusting the level of each subband is received, the method further includes the step of partially changing the shape of the selected icon according to the adjusted level of each subband, wherein the shape of the icon is a size or a color. It may be one or more of brightness.

According to an exemplary embodiment of the present disclosure, the method may further include remixing the mixed signal by adjusting a subband level of a source signal corresponding to the selected icon when a command regarding level adjustment for each subband is received.

According to the present invention, if one of the icons is selected, the method may further include displaying an ambience adjustment screen of a source signal corresponding to the selected icon, wherein the ambience adjustment screen may be a screen displaying a relative position of each source signal. .

According to the present invention, the method further comprises changing a shape of the selected icon according to the adjusted ambience when a command regarding ambience adjustment is received, wherein the shape of the icon includes one or more of size, color, and brightness. It may be.

According to the present invention, if a signal relating to an ambience adjustment command is received, adjusting the relative position of the source signal corresponding to the selected icon, and further comprising remixing the mixed signal, wherein the relative position is different from another source. It may be determined according to the location of the signal and the virtual user location.

According to another aspect of the invention, the input unit for selecting one of the plurality of mixed signals; A control unit which displays the source signal information corresponding to the mixed signal selected by the input unit, and displays an icon corresponding to the source signal based on the source signal information; And a display unit for displaying an icon corresponding to the source signal.

According to the present invention, the memory unit may further include a memory unit in which image data for each icon is stored, and the icon may be image data stored in the memory unit.

According to another aspect of the invention, the step of receiving a playback command for the mixed signal from the user; And changing the shape of the image of the source signal over time as the level of the source signal corresponding to the mixed signal changes from time to time, wherein the shape of the image includes: size, color, brightness, screen A method of displaying an interface of a mix signal including one or more of the locations is provided.

According to the present invention, the level of the source signal is a level for each subband of the source signal, and the displaying may be a step of partially changing the shape of the image of the source signal.

According to the present invention, the displaying step, the interface display method of the mixed signal, characterized in that the size of the image of the source signal other than the source signal with the highest level at a specific time point is zero.

According to the present invention, the image may be matched with metadata corresponding to the mixed signal.

According to another aspect of the invention, the input unit for inputting a playback command for the mixed signal; A control unit which controls to change the shape of the image of the source signal over time as the level of the source signal corresponding to the mix signal changes over time; A display unit for displaying an image corresponding to the source signal is provided, and the shape of the image is provided with an interface display device for a mix signal including at least one of size, color, brightness, and in-screen position.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms or words used in the specification and claims should not be construed as having a conventional or dictionary meaning, and the inventors should properly explain the concept of terms in order to best explain their own invention. Based on the principle that can be defined, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention. Therefore, the embodiments described in the specification and the drawings shown in the drawings are only the most preferred embodiment of the present invention and do not represent all of the technical idea of the present invention, various modifications that can be replaced at the time of the present application It should be understood that there may be equivalents and variations.

In particular, the mixed signal in the present invention includes an audio signal, a video signal, and the like, and should be understood as a concept including the composite object signal of the present specification.

1 is a block diagram of a first remix signal encoder according to an embodiment of the present invention. The first remix signal encoder includes an additional information generator 103 and an additional information encoder 105.

Referring to FIG. 1, the additional information generator 103 generates additional information 104 using a typical mix signal 101 and a source signal 102 constituting the mixed signal. The mix signal 101 may be a mono, stereo, and multi-channel audio signal. The source signal 102 may be some or all of the source signals constituting the mix signal 101. The additional information 104 refers to information used to process the mixed signal in source signal units. The additional information 104 includes mix parameters for remixing the mix signal. The mix parameter may include an encoder mix parameter generated using a source signal in an encoder, and optionally, a blind mix parameter generated using only a mix signal. Examples of the mix parameter may include a gain value and a subband power for each source signal. A detailed definition and generation method for the additional information 104 is described in FIG. 2. The present invention also includes generating the additional information 104 using only the source signal 102 constituting the mix signal. The side information encoding unit 105 encodes the generated side information 104 to generate an encoded side information signal 106. The mix signal 101 and the side information signal 106 are transmitted to a decoding apparatus.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when the mixed signal is a stereo signal. As described above, the mixed signal used in the present invention may be a mono, stereo, and multi-channel audio signal, but for convenience, it will be described with reference to the stereo signal 201.

및

는 상기 스테레오 신호를 구성하는 소스 신호들의 합으로 표현될 수 있다. 여기서, n은 타임 인덱스를 의미한다. 따라서, 상기 스테레오 신호(201)는 아래의 [수학식 1]과 같이 표 현될 수 있다.The stereo signal 201

And

May be expressed as a sum of source signals constituting the stereo signal. Here, n means time index. Therefore, the stereo signal 201 may be expressed as shown in Equation 1 below.

은 소스 신호들을 나타낸다.

및

는 각각의 소스 신호에 대한 진폭 패닝(amplitude panning) 및 게인(gain)을 결정하는 값이다. 모든

들은 서로 독립적이다. 상기

는 모두 순수한 소스 신호이거나, 또는 순수한 소스 신호에 약간의 잔향(reverberation) 및 효과음 신호성분(sound effect signal components)을 포함할 수 있다. 예를 들면, 특정한 잔향 신호성분은 2개의 소스 신호, 즉, 왼쪽 채널로 믹스된 신호와 오른쪽 채널로 믹스된 신호로 표현될 수 있다.Here, I is the number of source signals included in the stereo signal,

Represents source signals.

And

Is a value that determines the amplitude panning and gain for each source signal. all

They are independent of each other. remind

Are all pure source signals, or may include some reverberation and sound effect signal components in the pure source signal. For example, the specific reverberation signal component may be represented by two source signals, that is, a signal mixed with the left channel and a signal mixed with the right channel.

It is an object of the present invention to modify a stereo signal comprising the source signal such that M (0 <= M <= I) source signals are remixed. The source signals may be remixed into a stereo signal with different gain factors. The remix signal may be expressed as Equation 2 below.

및

는 리믹스되는 M개의 소스 신호들에 대한 새로운 게인 팩터들이다. 상기

및

는 디코더 단에서 제공될 수 있다. 이 경우에, 부가정보생성부(206)는 스테레오 신호(201) 및 M개의 소스 신호(202)를 이용하여 부가 정보(207)를 생성할 수 있다. here,

And

Are new gain factors for the M source signals to be remixed. remind

And

May be provided at the decoder stage. In this case, the additional information generator 206 may generate the additional information 207 using the stereo signal 201 and the M source signals 202.

에 대한 접근 없이, [수학식 1]로 표현되는 통상적인 믹스 신호가 주어지는 경우에, [수학식 2]로 표현되는 리믹스 신호를 지각적으로 모방하는 것을 목적으로 한다.As described above, an object of the present invention is to remix the stereo signal in the unit of the source signal when given a conventional stereo signal and some additional information. As in the present invention, it is not possible to completely generate the remix signal represented by Equation 2 from the mixed signal represented by Equation 1 using a very small amount of additional information. Thus, the present invention provides the respective source signals

It is aimed to perceptually mimic the remix signal represented by [Equation 2], given an ordinary mix signal represented by [Equation 1] without access to.

및

로 표시된다. 여기서 k는 서브밴드 신호들의 시간 인덱스(time index)이다. 유사하게, M개의 소스 신호들(202)의 서브밴드 신호들(205)은

,

,...,

로 표시된다. 명료한 표현을 위해, 서브밴드(주파수) 인덱스를 사용하지 않았다.Referring to FIG. 2, a stereo signal 201 and M source signals 202 included in the stereo signal 201 are input to a first remix signal encoding apparatus. The stereo signal 201 may be delayed to some extent to be synchronized with the side information and used directly as an output signal. To generate the additional information, the stereo signal 201 and the source signals 202 are decomposed into subband-specific signals 204 and 205 in the time-frequency domain through the filter bank 203. That is, the stereo signal 201 and the source signal are processed in the time-frequency domain, which will be described later with reference to FIG. 3. The subband-specific signal 204 is similarly processed at the center frequency of each subband. At a particular frequency, the subband pair 204 of the stereo signal 201

And

Is displayed. Where k is the time index of the subband signals. Similarly, the subband signals 205 of the M source signals 202 may be

,

, ...,

Is displayed. For clarity, no subband (frequency) index is used.

를 생성한다. 또한, 상기 부가정보생성부(206)는 스테레오 신호(201)의 서브밴드 쌍(204)을 이용하여, 서브밴드별로 게인 팩터

및

를 생성한다. 상기 게인 팩터

및

는 외부에서 직접 주어질 수 있다. 상기 서브밴드별 숏-타임 서브밴드 파워 및 게인 팩터를 이용하여 서브밴드별 부가 정 보(207)가 생성된다. 상기 부가정보생성부(206)는 상기 숏-타임 서브밴드 파워 및 게인 팩터들 이외에 상기 스테레오 신호에 관련된 다른 정보를 부가 정보(207)로 생성할 수 있다. 부가정보인코딩부(208)는 상기 서브밴드별 부가정보(207)를 이용하여 부호화된 부가 정보 신호(209)를 생성한다. Given the subband signals 205 of the source signals 202, the side information generator 206 may perform short-time subband power for each subband,

Create In addition, the additional information generator 206 uses a subband pair 204 of the stereo signal 201 to obtain a gain factor for each subband.

And

Create The gain factor

And

Can be given directly from the outside. Additional information 207 for each subband is generated using the short-time subband power and gain factor for each subband. The additional information generator 206 may generate other information related to the stereo signal as additional information 207 in addition to the short-time subband power and gain factors. The additional information encoding unit 208 generates an additional information signal 209 encoded using the additional information 207 for each subband.

및

는 고정적이 될 것이다. 만일

및

가 시간 k에 따라 가변적이라면, 상기 게인 팩터들은 시간의 함수로 생성될 것이다. 상기 게인 팩터들은 직접 양자화 및 부호화되지 않고, 먼저 양자화 및 부호화에 더 적합한 다른 값들로 전환될 수 있다. 또한,

는 스테레오 신호(201)의 서브밴드 파워에 상대적인 값으로 정규화될 수 있다. 이것은 스테레오 신호를 효율적으로 부호화하기 위해 통상적인 인코딩 장치가 이용되는 경우에, 본 발명을 상대적으로 변화에 강하도록 만들어준다. 예를 들면,

및

는 아래의 [수학식 3]으로 표현되는 게인 및 데시벨(dB) 단위의 레벨차로 전환되어 전송될 수 있다. For many stereo signals 201, the gain factor

And

Will be fixed. if

And

If is variable over time k, the gain factors will be generated as a function of time. The gain factors are not directly quantized and coded, but may first be converted to other values more suitable for quantization and coding. Also,

Can be normalized to a value relative to the subband power of the stereo signal 201. This makes the present invention relatively resistant to changes when conventional encoding devices are used to encode stereo signals efficiently. For example,

And

May be converted into a level difference in units of gain and decibel (dB) expressed by Equation 3 below, and then transmitted.

는 부가 정보로서 직접 부호화되는 것이 아니라, 아래의 [수학식 4]로 표현되는 스테레오 신호에 상대적으로 정의된 값으로 변환되어 전송될 수 있다. Also,

Is not directly encoded as additional information, but may be converted into a value defined relative to the stereo signal represented by Equation 4 below and transmitted.

는 아래의 [수학식 5]와 같이 계산될 수 있다. To produce short-time subband power, the present invention uses single-pole averaging. In other words,

May be calculated as shown in Equation 5 below.

Here, α∈ [0,1] determines the time-constant of the estimation window (estimation window) which decreases exponentially as shown in Equation 6 below.

는 서브밴드 샘플링 주파수를 나타낸다. 예를 들면, T=40 ms를 이용할 수 있다. 이하에서, 는 숏-타임 평균(short-time averaging)을 나타낸다. 만일

및

가 주어지지 않는다면, 상기

및

는 부가정보생성부(206)에서 생성될 필요가 있다.

이므로,

는 아래의 [수학식 7]과 같이 계산된다.here,

Denotes a subband sampling frequency. For example, T = 40 ms can be used. In the following, Denotes short-time averaging. if

And

If is not given,

And

Needs to be generated in the additional information generation unit 206.

Because of,

Is calculated as shown in Equation 7 below.

는 아래의 [수학식 8]과 같이 계산된다.Similarly,

Is calculated as shown in Equation 8 below.

3 illustrates a domain for processing an audio signal according to an embodiment of the present invention. As mentioned above, the audio signal and the additional information are processed as subband-specific signals in the time-frequency domain as shown in FIG. Subband-specific signals in the time-frequency domain are perceptually derived. For example, a signal for each subband may be generated by using a short time fourier transform (STFT) having a sine analysis and synthesis window having a length of about 20 ms. In this case, the STFT coefficients may be grouped such that one group has a bandwidth that is about twice the equivalent rectangular bandwidth (ERB).

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention. The second remix signal encoding apparatus includes a downmixing unit 402, an additional information generating unit 403, and an additional information encoding unit 406.

Referring to FIG. 4, the downmixing unit 402 generates a sum signal 404 by adding a plurality of source signals 401. Unlike the first remix signal encoding apparatus, the second remix signal encoding apparatus transmits the sum signal 404 instead of the stereo signal. The additional information generator 403 generates the additional information 405 using the source signals 401. The additional information 405 includes subband power and gain factor corresponding to each source signal. In addition, the additional information 405 may include a parameter corresponding to a delay in the remix renderer. Similar to the first remix signal encoding apparatus, the additional information 405 may be converted into another value more suitable for quantization and encoding and transmitted. The additional information encoding unit 406 generates an encoded additional information signal 407 using the generated additional information 405. The generated sum signal 404 and the side information signal 407 are transmitted to the decoding apparatus. The present invention also includes an encoding apparatus that does not have a downmixing unit 402. In this case, the source signals 401 are not converted to the sum signal 404, and each source signal 401 is transmitted directly.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention. The first remix signal decoding apparatus includes a side information decoder 503 and a remix renderer 505.

Referring to FIG. 5, the mix signal 501 and the additional information signal 502 are input to the first remix signal decoding apparatus. The mix signal 501 may be a mono, stereo or multichannel audio signal. The additional information decoding unit 503 decodes the additional information signal 502 to generate additional information 504. The additional information 504 includes a gain factor and subband power of source signals included in the transmitted audio signal 501. The remix renderer 505 may input a user-mix parameter 506 generated using control information directly provided by the user. The remix renderer 505 generates the remix signal 507 using the mix signal 501, the transmitted additional information 504, and the user mix parameter 506. A detailed description of the method of generating the remix signal will be described later with reference to FIG. 6. The remix signal 507 is generated as an Eq-channel mix signal having the same number of channels as the number of channels of the transmitted mix signal, or an upchannel mix signal having more channels than the number of channels of the mix signal. (Up-channel mix signal) can be generated.

FIG. 6 is a detailed diagram of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. As described above, the transmitted mixed signal may be a mono, stereo, and multichannel audio signal, but for convenience, the stereo signal 601 will be described.

및

로 표현된다. 부가정보디코딩부(605)는 전송된 부가 정보 신호(602)를 복호화하여, 서브밴드별 부가 정보(606)를 생성한다. 또한, 리믹스 렌더링부(607)에 사용자가 제공하는 제어 정보를 이용하여 생성된 사용자 믹스 파라미터(608)가 입력될 수 있으며, 상기 사용자 믹스 파라미터(608)는 서브밴드별로 제공될 수 있다. 전술한 것처럼, 상기 부가 정보(606)는 리믹스 될 M개의 소스 신호에 대한 서브밴드별 게인 팩터(

및

) 및

로 표현되는 서브밴드 파워를 포함한다. 리믹스 렌더링부(607)는 서브밴드별로 생성된 스테레오 신호(604), 전송된 부가 정보(606) 및 사용자 믹스 파라미터(608)를 이용하여, 서브밴드별 리믹스 신호(609),

및

를 생성한다. 상기 리믹스 신호(609)를 생성하는 방법은 아래에서 더욱 상세하게 기술된다. 상기 리믹스 신호(609)는 역필터뱅크(610)를 통해 시간 도메인(time domain)의 스테레오 신호(611),

및

로 변환된다. Referring to FIG. 6, the stereo signal 601 is decomposed into a subband-specific signal 604 in the time-frequency domain through the filter bank 603. As shown in FIG. 6, the subband-specific signal 604 at a particular frequency

And

It is expressed as The additional information decoding unit 605 decodes the transmitted additional information signal 602 to generate additional information 606 for each subband. In addition, a user mix parameter 608 generated using control information provided by a user may be input to the remix renderer 607, and the user mix parameter 608 may be provided for each subband. As described above, the additional information 606 is a subband-specific gain factor for M source signals to be remixed.

And

) And

It includes the subband power represented by. The remix rendering unit 607 may use the stereo signal 604 generated for each subband, the additional information 606 transmitted, and the user mix parameter 608, and the remix signal 609 for each subband,

And

Create The method of generating the remix signal 609 is described in more detail below. The remix signal 609 is a stereo signal 611 in the time domain through the inverse filter bank 610,

And

Is converted to.

는 서브밴드별 소스 신호

로 교체된다. 즉, 서브밴드별 믹스 신호(604)는 아래의 [수학식 9]과 같이 표현될 수 있다. A method of generating the remix signal 609 generated by the remix renderer 607 is as follows. Equations 1 and 2 are also valid for the signals 604 and 609 for each subband. In this case, the source signal

Is the subband-specific source signal

Is replaced by. That is, the mix signal 604 for each subband may be expressed by Equation 9 below.

The subband remix signal 609 may be expressed by Equation 10 below.

및

가 주어지면, 아래의 [수학식 11]와 같이 서로 다른 게인들을 가지는 서브밴드별 리믹스 신호(609)가 상기 서브밴드별 믹스 신호(604)의 선형 조합으로 추정될 수 있다.To generate the remix signal 609, least squares estimation may be used. Per subband mix signal 604,

And

Given by Equation 11 below, the subband remix signal 609 having different gains may be estimated as a linear combination of the subband mix signals 604.

,

,

및

는 가중 팩터들(weighting factors)이다. 이때, 생성되는 추정 에러(estimation error)는 아래의 [수학식 12]과 같이 정의될 수 있다.here,

,

,

And

Is the weighting factors. In this case, the generated estimation error may be defined as shown in Equation 12 below.

,

,

및

는 평균제곱오차(mean square error),

및

가 최소가 되도록 서브밴드별로 생성될 수 있다. 이때, 추정 에러,

및

가

및

에 직교(orthogonal)될 때, 상기 평균제곱오차가 최소가 된다는 것을 이용할 수 있다. 생성되는

및

는 아래의 [수학식 13]과 같 이 표현될 수 있다.The weighting factors,

,

,

And

Is the mean square error,

And

May be generated for each subband such that is minimized. In this case, the estimation error,

And

end

And

When orthogonal to, it can be used that the mean square error is minimized. Generated

And

Can be expressed as Equation 13 below.

,

및

는 직접 생성될 수 있지만,

및

은 전송된 부가 정보(606)(예를 들면,

,

,

) 및 사용자가 제공하는 제어 정보(608)(예를 들면, 게인 팩터

및

)를 이용하여, 아래의 [수학식 14]와 같이 생성될 수 있다.here,

,

And

Can be generated directly,

And

Is transmitted additional information 606 (e.g.,

,

,

) And control information 608 provided by the user (e.g., gain factor

And

) Can be generated as shown in Equation 14 below.

및

가 아래의 [수학식 15]와 같이 생성될 수 있다. Similarly,

And

May be generated as shown in Equation 15 below.

및

는 아래의 [수학식 16]과 같이 표현될 수 있다. here,

And

May be expressed as Equation 16 below.

If the phases of the mix signal 604 are coherent or nearly synchronized with each other, a value expressed by Equation 17 below approaches 1.

In this case, the weights may be expressed as Equation 18 below.

및

)를 이용하여 각각의 소스 신호를 독립적으로 리믹스하여 생성된 리믹스 신호와 유사하게 들린다.The subband remix signal 609 generated as described above is converted into the remix signal 611 of the time-domain through the inverse filter bank 610 as described above. The remix signal 611 may generate a user mix parameter generated using control information provided by a user.

And

It sounds similar to the remix signal generated by independently remixing each source signal with

So far, the focus has been on remixing two-channel stereo signals. However, as described above, the present invention is not limited to stereo signals, but may be extended to remixing multichannel audio signals, for example, 5.1 channel audio signals. Those skilled in the art can remix multichannel audio signals, similar to the stereo signals described herein. In this case, Equation 11 may be written as Equation 19 below.

Optionally, one of the channels of the mix signal can be left without remixing. For example, for a 5.1 surround channel, it is possible to apply remixing only to the front channel without modifying the two back channels. In this case, a two or three channel remix algorithm is applied to the front channel.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention. The second remix signal decoding apparatus includes an additional information decoding unit 703, a spatial information integration unit 705, and a remix rendering unit 707.

Referring to FIG. 7, the sum signal 701 of the source signals and the additional information signal 702 are input to the second remix signal decoding apparatus. The additional information decoding unit 703 decodes the additional information signal 702 to generate additional information 704. The additional information 704 includes a gain factor, a delay constant, a subband power, and the like. The additional information integrator 705 separates the sum signal 701 into a plurality of source signals 706 by using the additional information 704. The remix renderer 707 may generate the remix signal 709 using the source signals 706. In this case, the remix renderer 707 may generate the remix signal 709 using the mix parameter transmitted as additional information. In addition, the remix renderer 707 may selectively generate the remix signal 709 using the user mix parameter 708 generated by using control information provided by the user.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention. The mixed signal 801 may be encoded by the conventional encoding apparatus 803 and converted into the encoded mixed signal 805. The mix signal 801 may be a channel-specific signal or a source signal. The conventional encoding device 803 includes not only conventional encoding devices such as AAC, MP3 encoder, etc., but also encoding devices to be developed in the future. The remix signal encoding apparatus 804 according to the present invention generates the additional information signal 806 using the mixed signal 801 and the source signal 802 included in the mixed signal. The multiplexer 807 generates the bitstream 808 using the encoded mix signal 805 and the side information signal 806. As described above, the additional information signal 806 may be inserted into an auxiliary data area in a conventional mix signal format to be compatible with conventional devices.

8B is a block diagram of a combination of a conventional decoding device and a remix signal decoding device according to an embodiment of the present invention. The demultiplexer 810 separates the encoded mix signal 811 and the side information signal 812 from the transmitted bitstream 809. The conventional decoding device 813 then decodes the encoded mix signal 811 to produce a mix signal 814 that can be used in the remix signal decoding device 815 according to the present invention. The conventional decoding device 813 includes not only a conventional encoding device such as an AAC, an MP3 decoder, etc. but also an encoding device to be developed in the future. The mix signal 814 may be a channel-specific signal or a source signal. The remix signal decoding apparatus 817 according to the present invention may convert the mix signal 814 into the remix signal 816 using at least one of the additional information signal 812 and the user mix parameter 817.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention. Referring to FIG. 9, the remix signal decoding apparatus includes a mixed signal decoder 901, a parameter generator 902, and a remix renderer 908. It may optionally include an effector (911). The parameter generator 902 may include a blind mix parameter generator 903, a user mix parameter generator 904, and a remix parameter generator 905. The remix parameter generation unit 905 may include an equimix parameter generation unit 906 and may optionally include an upmix parameter generation unit 907. In addition, the remix renderer 908 may include an equimix renderer 909 and optionally include an upmix renderer 910.

The mixed signal decoding unit 901 decodes the encoded mixed signal transmitted from the encoding end to generate a mixed signal. The parameter generator 902 receives additional information and user control information (or configuration information) transmitted from an encoding stage. The user control information is not transmitted at the encoder stage but may be generated at the decoder stage. The user mix parameter generator 904 generates a user mix parameter by using user control information. The additional information transmitted from the encoder stage may include an encoder mix parameter. In addition, the blind mix parameter generator 903 may generate a blind mix parameter using the mix signal. The encoder mix parameter and the blind mix parameter are alternatively input to the remix parameter generator 905.

The remix parameter generator 905 generates a remix parameter using additional information and a user mix parameter. The remix parameter may be generated to be applied to a channel of the remix signal. Equimix parameter generation unit 906 included in the remix parameter generation unit 905 generates a remix parameter used to generate a remix signal having the same channel number as the number of channels of the mix signal, the remix parameter generation unit 905 The upmix parameter generation unit 907, which may be included in the control unit, generates a remix parameter used to generate a remix signal having a larger number of channels than the number of channels of the mixed signal. The remix parameter is input to the remix renderer 908.

Equimix renderer 909 included in the remix renderer 908, using the remix parameter and the mix signal, having an equal number of channels equal to the number of channels of the mix signal (Eq-channel remix signal ) The upmix renderer 910, which may be included in the remix renderer 908, uses more channels than the number of channels of the mix signal by using the remix parameter and the mix signal generated by the upmix parameter generator 907. Generate an up-channel remix signal having a number. The upmix renderer 910 may generate an upchannel remix signal using the remix signal generated by the Equichannel renderer 909.

Accordingly, the decoding apparatus may output the mixed signal transmitted from the encoding stage as it is, output as an equal channel remix signal, or output as an up channel remix signal. Optionally, the remix renderer may give various effects to the remixed signal by using information provided from the effector 911.

10 is a diagram illustrating a configuration of an interface display device according to an embodiment of the present invention. Referring to the drawings, the interface display device 1100 according to an embodiment of the present invention includes an input unit 1110, a memory unit 1120, a controller 1130, and a display 1140, and a first remix signal decoding apparatus. It is associated with 500. Here, the first remix signal decoding apparatus 500 includes a side information decoding unit 503 and a remixing rendering unit 506. The first remix signal decoding apparatus 500 has the same structure and function as the first remix signal decoding apparatus described with reference to FIG. 5. Since it is so, the detailed description is omitted.

The input unit 1110 is an input device for selecting one of a plurality of mixed signals. Here, the mixed signal may be the mixed signal (audio signal for each channel) 501 described above with reference to FIG. 5. The mixed signal 501p selected by the input unit 1110 and the additional information signal 502p of the mixed signal 501p are input to the first remix signal decoding apparatus 500 among the plurality of mixed signals. The input unit 1110 may also be for inputting a command for adjusting the level of the source signal, a command for adjusting the level for each subband of the source signal, and a command for adjusting the ambience of the source signal. The input unit 1110 may be implemented not only by a pointing device such as a mouse, a trackball, a touch pad, but also by a direction movement key and a touch wheel, and of course, may be implemented by a remote controller using wireless communication such as infrared communication.

The memory unit 1120 is a storage device that stores image data for each icon. More specifically, the information may be stored together with a category and a keyword corresponding to the icon. Image data can be used to display icons as images as well as text.

The controller 1130 inquires the source signal information si corresponding to the mixed signal 501p selected by the input unit 1110, and then displays an icon corresponding to the source signal based on the source signal information si. . Here, the source signal information si is information about a source signal included in the mixed signal 501p selected by the input unit 1110 and may be information including an instrument name of each source signal. In addition, the source signal information si may be information of the source signal 102 described above with reference to FIG. 1. On the other hand, the source signal information si may further include meta data (instrument player name, singer name, etc.). The source signal information si may be received by the controller 1130 from the additional information decoder 503 of the first remix signal decoding apparatus 500.

Meanwhile, the icon corresponding to the source signal may be displayed as text, but may also be displayed as an image using image data stored in the memory unit 1120.

The control unit 1130 further includes various commands for remixing from the input unit 1110 (commands for level adjustment of the source signal, commands for level adjustment for each subband of the source signal, and commands for ambience adjustment of the source signal). And the like), the command is converted into the control information 506 and input to the remixing rendering unit 505. Since the control information 506 may be the same as the control information 506 described with reference to FIG. 5 and the control information 608 described with reference to FIG. 6, the detailed description thereof will be omitted.

The display 1140 is a display device in which an icon corresponding to a source signal is displayed under the control of the controller 1130.

11A to 11D illustrate examples of an interface display device according to an embodiment of the present invention. First, FIG. 11A illustrates an interface display device according to an embodiment of the present invention implemented as a mobile terminal integrally with the first remix signal decoding device 500. FIG. 11B illustrates an input device 1110 of the interface display device 1100 according to an embodiment of the present invention implemented as a mouse and a touch pen connected to the first remix signal decoding device 500 by wire. In the same way as the input device 1110, it can be seen that the monitor is connected to the first remix signal decoding apparatus 500 by wire. 11D illustrates that the display 1140 of the interface display device 1100 according to an embodiment of the present invention is connected to the first remix signal decoding device 500 by wire, but the input device 1110 decodes the first remix signal. The device 500 is a remote control for wireless communication. Referring to FIG. 11D, the input device 1110 of the interface display device 1100 according to an embodiment of the present invention is implemented as a remote controller as in the case of FIG. 11C, and the display 1140 is provided in the remote controller. Can be. As such, the interface display apparatus 1100 and the first remix signal decoding apparatus 500 according to an exemplary embodiment may be implemented in various forms.

12 is a diagram illustrating a configuration of an interface display device according to another exemplary embodiment of the present invention. Referring to the drawings, the interface display device 1200 according to another exemplary embodiment of the present invention includes an input unit 1210, a memory unit 1220, a controller 1230, and a display 1240, and a second remix signal decoding apparatus. (700). Here, the first remix signal decoding apparatus 700 includes the additional information decoding unit 703, the additional information integration unit 705, and the remixing rendering unit 708, and the second remix signal described with reference to FIG. 7 is described above. Since the structure and function of the decoding apparatus are almost the same, a detailed description thereof will be omitted.

Meanwhile, the interface display device 1200 according to another embodiment of the present invention is compared with the interface display device 1100 according to an embodiment of the present invention, instead of the first remix signal decoding device 500, the second remix signal. The only difference is that the composite object signal 701p is inputted instead of the mixed signal 501p, and the components 1210, 1220, 1230, and 1240 are connected to the decoding apparatus 700. Since the interface display device 1100 according to the embodiment is substantially the same as each of the components 1110, 1120, 1130, and 1140 of the same name, a description thereof will be omitted.

13A and 13B are diagrams illustrating a procedure of an interface display method according to an embodiment of the present invention. The term mixed signal used below should be understood as a concept including the concept of a composite object signal. The interface display method according to an embodiment of the present invention may be performed by the interface display device 1100 according to an embodiment of the present invention described above, or the interface display device 1200 according to another embodiment of the present invention. It may also be performed by.

First, referring to FIG. 13A, the interface display device displays information on one or more mix signals that are already stored or can be transmitted as a list of mix signals in operation S110. Then, the user selects a specific mix signal to be reproduced or to know the source signal information from the list of the mixed signals displayed in step S120 by using the input device (step S120). Then, the interface display device extracts source signal information corresponding to the mixed signal from the additional information corresponding to the mixed signal selected in step S120 (step S130). When metadata of the mixed signal is included in the source signal information extracted in step S130, metadata is extracted from the source signal information (step S140).

Then, the interface display device inquires an icon corresponding to the source signal based on the source signal information extracted in step S130 (step S150). Here, the icon may be composed of text or an image. If metadata is extracted in operation S140, an image matching the metadata may be inquired. Then, the icon queried in step S150 is displayed on the display (step S160). An example of a screen on which an icon for each source signal of a specific mix signal is displayed is illustrated in FIG. 14A. FIG. 14A illustrates a screen in which icons corresponding to a piano, a violin, a singer (woman), and a flute are displayed when the source signal information is “piano, violin, singer (woman), flute”. As described above, the interface display method according to the exemplary embodiment of the present invention displays an icon for a specific mix signal through steps S110 to S150. An example of a screen in which an icon for each source signal of a specific mix signal expresses metadata is illustrated in FIG. 14B. 14B illustrates that when the metadata included in the source signal information is "piano performer: Jung Myung-hoon" and "vocalist: assistant S.", the icon corresponding to the piano is composed of a photograph of Jeong Myung-hoon, and the icon corresponding to the vocal music is the image of Jo Sumi. It is a screen composed of photos. Hereinafter, a process will be described after a specific icon is executed among the icons displayed through the above process.

Referring to FIG. 13B, when a user selects one of the icons displayed in step S150 and inputs an execution command for a specific icon, the interface display device receives an execution command for the selected icon (step S210). The execution command of the icon, that is, the process performed when the icon is executed can be various depending on the setting. The following three cases (level adjustment, equalizer adjustment, and ambience adjustment) will be described.

If the execution command of the icon is set to level adjustment (volume adjustment) (YES in step S220), the level adjustment screen of the source signal corresponding to the icon executed in step S210 is displayed (step S222). An example of the level adjustment screen is shown in FIG. 15A. FIG. 15A illustrates a screen in which a control bar for adjusting the level (volume) of the flute is displayed when the "flute" icon is executed among four icons. On a screen as shown in FIG. 15A, a user may input a command for adjusting a level of a specific source signal (eg, flute) by using a pointing device to move a control bar or using a direction key of a keypad. When the command regarding the level adjustment is received (YES in step S224), the shape of the icon is changed according to the adjusted level (step S226). Here, the shape of the icon is a concept including one or more of size, color, and brightness. An example of a screen in which the shape of the icon is changed according to the adjusted level is shown in FIG. 15B. Referring to FIG. 15B, as the level of the flute is adjusted to be high, it can be seen that the size of the icon corresponding to the flute is increased. Then, the level adjustment command input in step S224 is converted into control information, thereby remixing the mixed signal selected in step S120 (step S250). Here, the control information is information input to the first remix signal decoding apparatus or the second remix signal decoding apparatus. Since the control information is described together with the interface display apparatus according to the exemplary embodiment of the present invention, description thereof will be omitted.

If the execution command of the icon is set to equalizer adjustment (level adjustment for each subband) (YES in step S230), the level adjustment screen of the source signal corresponding to the icon executed in step S210 is displayed (S232). step). An example of an equalizer screen (subband level adjustment screen) is shown in FIG. 16. FIG. 16 illustrates a screen in which a control bar for each subband is displayed so that an equalizer of a piano may be adjusted when a “piano” icon is executed among four icons. On the screen as shown in FIG. 16, a user may input a command for adjusting a subband level of a specific source signal (for example, a piano) by moving a control bar for each subband using a pointing device. When the command regarding the level adjustment for each subband is received (YES in step S234), the shape of the icon is partially changed according to the adjusted level for each subband (step S236). For example, when the level of the low frequency band is adjusted high, the brightness of the portion corresponding to the left side of the icon may be darkened. Thereafter, the level adjustment command for each subband input in step S234 is converted into control information, thereby remixing the mixed signal selected in step S120 (step S250).

If the execution command of the icon is set to ambience adjustment (relative position adjustment) (YES in step S240), the ambience adjustment screen (relative position adjustment) of the source signal corresponding to the icon executed in step S210 is displayed. (Step S242). An example of an ambience adjustment screen (relative position adjustment screen) is shown in FIG. FIG. 17 is a screen showing a relative position of a piano so as to adjust the ambience of the piano when the “piano” icon of the four icons is executed. On the screen as shown in FIG. 17, the user may input a command for adjusting an ambience of a specific source signal (for example, piano) by moving an icon using a pointing device or a direction key. When such a command regarding ambience adjustment is received (YES in step S244), the shape of the icon is changed according to the adjusted level for each subband (step S246). For example, when the relative position of the adjusted source signal approaches the position of the user (listener), the brightness of the icon may be brightened. Then, by converting the ambience adjustment command input in step S244 into control information, the mixed signal selected in step S120 is remixed (step S250).

18 is a configuration diagram of an interface display device according to still another embodiment of the present invention. Referring to FIG. 18, an interface display device 2100 according to another embodiment of the present invention includes an input unit 2110, a memory unit 2120, a controller 2130, and a display 2140, and a first remix signal. It is associated with the decoding device 500. Here, the first remix signal decoding apparatus 500 includes a side information decoding unit 503 and a remixing rendering unit 506. Also, the first remix signal decoding apparatus 500 includes a structure and a function of the first remix signal decoding apparatus described above with reference to FIG. 5. Since this is almost the same, a detailed description thereof will be omitted.

The input unit 2110 is an input device for inputting a playback command for the mixed signal. The input unit 2110 may be implemented not only by a pointing device such as a mouse, a trackball, a touch pad, but also by a direction moving key and a touch wheel.

The memory unit 2120 is a storage device in which an image for each source signal is stored. For example, an image for each instrument (piano image, violin image, flute image, etc.), and a player-specific image (Jung Myung-hoon photo, Jo Su-mi photo, etc.) may be stored.

The controller 2130 controls to change the shape of the image of the source signal over time as the level of the source signal corresponding to the mix signal changes over time. Here, the image is retrieved from the memory unit 2120 based on the source signal information si received from the additional information encoding unit 503 or the remixing rendering unit 505 of the first remix signal decoding apparatus 500. It may be an image for each source signal. The time level of the source signal may be extracted from the source signal level information sl received from the remixing rendering unit 505 of the first remix signal decoding apparatus 500.

For example, if the total playback time is 3 minutes 45 seconds, the source signal (piano, violin, flute) every second, from the start of playback (t = 0 minutes 0 seconds) to the completion of playback (t = 3 minutes 45 seconds) It detects each level and changes and displays the shape (size, color, brightness, position on the screen, etc.) of the corresponding image (piano image, violin image, flute image). If the level of the piano (or violin, etc.) is large at a specific point in time, the size of the piano image (or violin image, etc.) can be greatly changed. You can also change the brightness of your light (or violin).

The display 2140 is a display device for displaying an image corresponding to the source signal.

19 is a diagram illustrating a configuration of an interface display device according to another embodiment of the present invention. Referring to the drawings, the interface display device 2200 according to another exemplary embodiment of the present invention includes an input unit 2210, a memory unit 2220, a controller 2230, and a display 2240, and decodes a second remix signal. It is associated with the device 700. Here, the first remix signal decoding apparatus 700 includes the additional information decoding unit 703, the additional information integrating unit 705, and the remixing rendering unit 708. The second remix signal decoding apparatus 700 may also be described with reference to FIG. 7. Since the configuration and function of the remix signal decoding apparatus are almost the same, a detailed description thereof will be omitted.

Meanwhile, when compared to the interface display device 2100 according to another embodiment of the present invention, the interface display device 2200 according to another embodiment of the present invention may have a second remix instead of the first remix signal decoding device 500. The difference between the interface decoding apparatus 700 and the interface display apparatus 2100 according to an exemplary embodiment of the present invention is different from the fact that the synthesized object signal 701 is input instead of the mixed signal 501. Therefore, the description of the components will be omitted.

20 is a diagram illustrating a procedure of an interface display method according to another embodiment of the present invention. The term mixed signal used below should be understood as a concept including the concept of a composite object signal. The interface display method according to another embodiment of the present invention may be performed by the interface display device 2100 according to another embodiment of the present invention described above, or the interface display device according to another embodiment of the present invention ( 2200).

Referring to FIG. 20, when a user inputs a playback command for a specific mix signal by using an input device, the interface display device receives it (step S310). Then, the interface display device receives source signal information corresponding to the mixed signal (S320). The interface display device inquires an image corresponding to each source signal based on the source signal information received in operation S320 (operation S330).

Then, when the reproduction of the mixed signal is started (YES in step S340), the interface display device detects the level (or the level of each subband) of the source signal corresponding to the mixed signal over time (S350). Then, the shape (size, color, brightness, position in the screen, etc.) of the image queried in step S330 is displayed according to time according to the level (level for each subband) queried in step S340 (step S360). . 21 illustrates an example of a screen in which an image size changes in proportion to the level of the source signal. Referring to the size of the image of each object in FIG. 21, when the current playing time is 't1', the level of the female singer is high while the level of the instruments (piano, violin, flute) except the female singer is low. have. When the current playing time is 't2', only the female singer's level is lower than other instruments, and when the current playing time is 't3', only the piano and the female singer's level are high. An example of a screen in which the brightness of an image changes in proportion to the level of the source signal is shown in FIG. 21. Referring to FIG. 21, it can be seen that when the current times are t1, t2, and t3, the brightness of the image for each object changes as the level of the source signal changes.

Meanwhile, as the level of the source signal changes, the size of the image corresponding to the source signal may be changed. Only an image of the source signal having the highest level is displayed at a specific point in time, and the size of the image of the other source signals is zero. It may not be displayed on the screen. 23 and 24 show examples of a screen in which only an image having the current highest level among the source signals is displayed. Referring to FIG. 23, when the play time is t1 after the start of playback, the image of the other source signal (violin, vocalist, flute), which is the highest level of the piano among the four source signals, disappears and the piano image (piano player Jeong Myung-hoon) Only the image of the vocalist (female) is displayed on the screen when the playback time is t2. It can be seen. Referring to FIG. 24, when the playback time is t1 after the start of playback, the image of the third mantissa with the highest level is placed in the center, and when the playback time is t2, the image of the first mantissa with the highest level is centered. It can be seen that it is arranged.

If the image is changed according to the level of each subband of the source signal, the shape of the image of the source signal may be partially changed. For example, when the horizontal axis of the image corresponds to the frequency band axis of the source signal and the vertical axis of the image corresponds to the level axis of the source signal, when the level of the low frequency band is high and the level of the high frequency band is high, the left side of the image is faintly blurred. The right side of the image. The steps S350 to S360 continue until the reproduction of the mixed signal ends (step S370).

As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited thereto and is intended by those skilled in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalents of the claims to be described.

According to an aspect of the present invention, since information about a source signal (or an object signal) associated with a mixed signal is provided as a Graphic User Interface (GUI), the user can conveniently control the object, so that the user Can very easily remix the mix signal.

According to another aspect of the present invention, since each icon is configured based on the metadata of the source signals, the user can know who is the performer or singer of each object even if they only see the icon for each object, so that the information for each object It is very quick and easy to check.

According to another aspect of the present invention, while the mix signal is reproduced, as the shape of the image representing each source signal changes over time, the user may check each level (or level per subband) for each source signal in real time. Not only can you accurately determine if it's been remixed as you want, it's also fun to watch the media.

Claims (20)

Receiving a mix signal including one or more source signals from a user; Querying source signal information corresponding to the mixed signal; Displaying one or more icons corresponding to the one or more source signals based on the source signal information; And, And displaying a level adjustment screen of a source signal corresponding to the one or more source signals. The method of claim 1, And the source signal information is extracted from additional information corresponding to the mix signal. The method of claim 1, Extracting metadata corresponding to the mixed signal; And, The icon is an interface display method of the mixed signal, characterized in that the image matching the metadata. The method of claim 3, wherein And the metadata is extracted from additional information corresponding to the mixed signal, and includes at least one of a musical instrument player name and a vocal artist name of the source signal. The method of claim 1, And the level adjustment screen corresponds to a source signal corresponding to the selected icon when one of the one or more icons is selected. The method of claim 1, Changing a shape of an icon corresponding to the one source signal according to the adjusted level when a command for level adjustment is received with respect to one of the one or more source signals; The shape of the icon, the interface display method of the mixed signal, characterized in that at least one of the size, color, brightness. The method of claim 1, If a command regarding a level adjustment is received for one source signal of the one or more source signals, adjusting the level of the one source signal, further comprising the step of remixing the mixed signal; . The method of claim 1, And if one of the one or more icons is selected, displaying an equalizer screen of the source signal corresponding to the selected icon. The method of claim 8, If the command regarding the level adjustment for each subband is received, further comprising partially changing the shape of the selected icon according to the adjusted level for each subband, The shape of the icon, the interface display method of the mixed signal, characterized in that at least one of the size, color, brightness. The method of claim 8, And receiving a command relating to level adjustment for each subband, adjusting the subband level of the source signal corresponding to the selected icon, and remixing the mixed signal. . The method of claim 1, If one of the one or more icons is selected, further comprising displaying an ambience adjustment screen of a source signal corresponding to the selected icon, The ambience adjustment screen, the interface display method of the mixed signal, characterized in that the screen is displayed the relative position for each source signal. The method of claim 11, If a command regarding ambience adjustment is received, changing the shape of the selected icon according to the adjusted ambience, The shape of the icon, the interface display method of the mixed signal, characterized in that at least one of the size, color, brightness. The method of claim 11, Remixing the mixed signal by adjusting a relative position of a source signal corresponding to the selected icon when a signal related to an ambience adjustment command is received, And the relative position is determined according to a position of another source signal and a virtual user position. An input unit for selecting a mixed signal including one or more source signals; A control unit for querying source signal information corresponding to the mixed signal selected by the input unit, and displaying one or more icons corresponding to the one or more source signals based on the source signal information; And, And a display unit for displaying an icon corresponding to the one or more source signals and a level adjustment screen of the source signal corresponding to the one or more source signals. The method of claim 14, And the level adjusting screen corresponds to a source signal corresponding to the selected icon when one of the one or more icons is selected. Receiving a play command for a mix signal from a user; And, Changing the shape of the image of the source signal over time as the level of the source signal corresponding to the mixed signal changes over time, and displaying the changed shape over time. The shape of the image, the interface display method of the mixed signal characterized in that it comprises at least one of the size, color, brightness, in-screen position. The method of claim 16, The level of the source signal is a level for each subband of the source signal, The displaying step, And partially changing the shape of the image of the source signal. The method of claim 16, The displaying step, And the size of the image of the source signal other than the source signal having the highest level at a specific time point is zero. The method of claim 16, And the image is matched with metadata corresponding to the mix signal. An input unit for inputting a playback command for the mix signal; A control unit which controls to change the shape of the image of the source signal over time as the level of the source signal corresponding to the mix signal changes over time; A display unit for displaying an image corresponding to the source signal, And the shape of the image comprises at least one of a size, a color, a brightness, and an on-screen position.

Images