JP5081250B2 - Command input device and method, media signal user interface display method and implementation thereof, and mix signal processing device and method - Google Patents

Description

  The present invention relates to an input device and an input method, and more particularly, to an input device and an input method capable of inputting a command necessary for controlling a mix signal. The present invention also relates to a mixed signal interface display method and apparatus, and more particularly to a mixed signal interface display method and apparatus capable of providing an interface to a user regarding a mixed signal such as an audio signal and a video signal. The present invention also relates to a mixed signal processing method and apparatus.

  In general, stereo signals are most frequently generated and are most widely used by consumers. Recently, multi-channel signals are increasingly used. However, there is a limit that the mix signal is processed not in units of source signals constituting the mix signal but in units of channel signals. Therefore, when a mix signal is processed in units of channel signals, there is a problem that it is not possible to independently process only specific source signals constituting the mix signal. For example, when watching a movie, it is impossible to increase only the volume of background music while keeping the volume of actors' voices constant.

  In addition, the user may increase or decrease the volume level for each source signal in order to process and remix the mix signal (or synthesized object signal) separately for each source, that is, for each specific source signal ( Or mute the sound by setting the level to 0), or adjust the relative position of the source signal. However, conventional remote controllers can only adjust the volume level of the entire mix signal, and cannot adjust the volume level or ambience of each source signal included in the mix signal. is there.

  In addition, in order to remix a mix signal (or synthesized object signal) by source, that is, by source signal, the user has to know what kind of source signal is included in the corresponding mix signal. Don't be. For example, if the mix signal is a music file, the user must check whether the source signal associated with the music file is a piano, violin, flute, or the like. However, conventionally, only the information of the mix signal (or the synthesized object signal) is displayed, and there is a problem that the information of the source signal related to the mix signal is not displayed.

  The present invention has been devised to solve the above problems, and its purpose is to adjust the volume level for each source signal included in a mix signal such as a video signal and an audio signal. It is an object to provide an input device and an input method that can adjust the ambience (relative position).

  Another object of the present invention is to provide an interface display method for a mix signal that can be displayed so that a user can always check information on a source signal (or object signal) associated with the mix signal (or composite object signal), and To provide an apparatus.

  Still another object of the present invention is to provide a mixed signal interface that can provide information about a source signal (or object signal) associated with a mixed signal (or synthesized object signal) through a graphic user interface (GUI). To provide a display method and apparatus.

  Yet another object of the present invention is to provide an image representing a source signal associated with a mix signal (or composite object signal) while a mix signal (or composite object signal) such as an audio signal, video signal, etc. is played. An object of the present invention is to provide an interface display method and apparatus for a mix signal that can be expressed in various ways according to time.

  Still another object of the present invention is to provide a method of generating additional information for each source signal using a signal similar to the source signal when there is no other source signal included in the mix signal.

  It is still another object of the present invention to provide a method and apparatus for generating additional information for remixing a mix signal using the mix signal in a decoding apparatus.

  It is still another object of the present invention to provide a method and apparatus for converting a specific source signal included in a mix signal into a source signal provided by a user.

  It is still another object of the present invention to provide a signal processing method and apparatus that allows a user to transform a mix signal using a source signal.

  Still another object of the present invention is to provide a signal processing method and apparatus that allows a user to control a mix signal in units of source signals.

  Still another object of the present invention is to provide a signal processing method and apparatus that allow a user to receive desired remix information from a remix information providing server.

  Still another object of the present invention is to provide a signal processing method and apparatus capable of uploading remix information generated by a user to a remix information providing server.

  Still another object of the present invention is to provide a signal processing method and apparatus capable of receiving and generating a mix signal from which some source signals are excluded.

  In order to achieve the above object, an input device according to the present invention includes an input unit; a source corresponding to a mix signal by the source-specific menu display command after converting an input signal received through the input unit into a source-specific menu display command A control unit that outputs a menu display screen for each source based on information; and an input device including a display for outputting the menu display screen for each source.

  The present invention may further include a pointing device, and the controller may convert an input signal received through the pointing device into a source-specific level adjustment command.

  The present invention may further include a direction movement key, and the controller may convert an input signal received through the direction movement key into a source-specific level adjustment command.

  According to another aspect of the present invention, an input unit; a control unit that converts an input signal received through the input unit into a menu display command for each source; and a command for transmitting the command converted by the control unit to an external device An input device including a wireless communication unit is provided.

  According to the present invention, the wireless communication unit is for receiving data related to the menu screen by source from the external device, and may further include a display on which the menu screen by source is displayed.

  According to still another aspect of the present invention, an input unit; and a control unit that converts an input signal received through the input unit into an audio mute command and outputs control information related to the mix signal based on the audio mute command. An input device is provided.

  According to still another aspect of the present invention, an input unit; and a control unit that converts an input signal received through the input unit into an audio level adjustment command and outputs control information related to the mix signal based on the audio level adjustment command An input device is provided.

  According to still another aspect of the present invention, an input unit; and an input signal received through the input unit are converted into an ambience screen display command, and based on source information corresponding to a mix signal according to the ambience screen display command And an input device including a control unit for outputting the ambience screen; and a display for outputting the ambience screen.

  The control unit may further include a pointing device or a direction movement key, and the control unit may convert an input signal received through the pointing device into an ambience adjustment command, and output control information related to the mix signal based on the ambience adjustment command. can do.

  According to still another aspect of the present invention, an input unit includes: a control unit that converts an input signal received through the input unit into a source sound mute command and outputs control information related to the mix signal based on the source sound mute command The source sound mute instruction is provided as an instruction for preventing a specific source signal included in the mix signal from being reproduced.

  According to still another aspect of the present invention, receiving a shortcut key registration command; one key of a plurality of keys, one source of a plurality of sources, one function of one or more functions is selected. And matching the selected key to the selected function and storing the selected function with respect to the selected source, wherein the plurality of sources include an audio signal source and a video signal source. An input method is provided that includes one or more of the sources, and wherein the one or more functions include one or more of sound mute and ambience adjustment.

  According to the present invention, the step of selecting may be a step of selecting a number key from a plurality of number keys.

  According to still another aspect of the present invention, one key among a plurality of keys, one source among a plurality of sources, and an input unit for selecting one function among one or more functions; A storage unit storing instruction information; and a control unit that matches the selected function with the selected source by the input unit and stores the selected function in the storage unit, The plurality of sources may include one or more of an audio signal object and a video signal object, and the one or more functions may include one or more of sound mute and ambience adjustment. .

  The method for displaying an interface according to the present invention includes the steps of: selecting a specific mix signal by a user; and querying source signal information corresponding to the mix signal to display a list of the source signals on a display, The display is maintained and continues while the mix signal is selected.

  According to the present invention, the source signal information can be extracted from additional information corresponding to the mix signal.

  According to the present invention, the list of source signals may be one or more of an instrument name, a singer name, and a character name of the source signal included in the mix signal.

  According to the present invention, the method may further include the step of selecting one source from the list of source signals by the user; and further displaying the metadata regarding the selected source.

  According to the present invention, the metadata is extracted from the additional information corresponding to the mix signal, and may include one or more of the instrument player name and the production date and time of the source signal.

  According to the present invention, the method may further include a step of selecting one source from the list of source signals by a user; and further displaying a volume level adjustment screen for the selected source.

  According to the present invention, the method may further include a step of selecting one source from the list of source signals by a user; and displaying an equalizer adjustment screen for the selected source.

  According to the present invention, the method may further include selecting a source from the list of source signals by a user; and further displaying an ambience adjustment screen for the selected source.

  According to still another aspect of the present invention, an input unit for selecting one from a plurality of mix signals; a control unit that queries source signal information corresponding to the mix signals and displays a list of the source signals; And a display for displaying the list of source signals, wherein the display is maintained while the mix signal is selected, and an interface display device for the mix signal is provided.

  The display may further include a second display, the input unit may select one source from the list of source signals, and the control unit may adjust metadata and volume level adjustment for the selected source. One or more of a screen, an equalizer adjustment screen, and an ambience adjustment screen can be displayed on the second display.

  According to the present invention, the display can display the list of source signals by blinking a plurality of light emitting elements.

  According to the present invention, the display and the second display may be composed of one panel.

  The method for displaying an interface of a mix signal according to the present invention includes a step of selecting a specific mix signal by a user; and inquiring source signal information corresponding to the mix signal, and corresponding to the source signal based on the source signal information A step of displaying an icon to be performed.

  According to the present invention, the source signal information is extracted from additional information corresponding to the mix signal.

  According to the present invention, the icon may be an image to be matched with the metadata, including extracting metadata corresponding to the mix signal.

  According to the present invention, the metadata is extracted from the additional information corresponding to the mix signal, and may include one or more of the instrument player name and vocalist name of the source signal.

  The method may further include displaying a source signal level adjustment screen corresponding to the selected icon when an execution command for one of the icons is received.

  According to the present invention, when receiving a command for level adjustment, the method further includes changing the shape of the selected icon according to the adjusted level, wherein the icon shape is one of size, hue, and brightness. The above can be included.

  According to the present invention, the method may further include adjusting the level of the source signal corresponding to the selected icon and remixing the mix signal when receiving the level adjustment command.

  The method may further include displaying an equalizer screen of a source signal corresponding to the selected icon when one of the icons is selected.

  According to the present invention, when receiving an instruction for level adjustment by subband, the method further includes partially changing the shape of the selected icon according to the adjusted level by subband. , Hue, and brightness.

  The method may further include adjusting a subband level of a source signal corresponding to the selected icon and remixing the mixed signal when receiving a command for level adjustment by subband.

  According to the present invention, when one of the icons is selected, an ambience adjustment screen of a source signal corresponding to the selected icon is further displayed, and the ambience adjustment screen includes a relative position for each source signal. Can be displayed on the screen.

  According to the present invention, when receiving an instruction regarding ambience adjustment, the method further includes changing a shape of the selected icon according to the adjusted ambience, wherein the icon shape is one of size, hue, and brightness. The above can be included.

  According to the present invention, when receiving a signal related to an ambience adjustment command, the method further includes adjusting a relative position of a source signal corresponding to the selected icon and remixing the mix signal, the relative position including: It can be determined by the position of other source signals and the virtual user position.

  According to still another aspect of the present invention, an input unit for selecting one of a plurality of mix signals; after inquiring source signal information corresponding to the mix signal selected by the input unit, the source signal information And a display unit for displaying an icon corresponding to the source signal, and a display unit for displaying the icon corresponding to the source signal.

  According to the present invention, the image processing apparatus may further include a storage unit that stores image data for each icon, and the icon may be image data stored in the storage unit.

  According to still another aspect of the present invention, receiving a playback command for a mix signal from a user; and changing the shape of the source signal image according to time as the level of the source signal corresponding to the mix signal changes with time. There is provided a method for displaying an interface of a mixed signal, including a step of changing and displaying, wherein the shape of the image includes one or more of a size, a hue, a brightness, and a position in a screen.

  According to the present invention, the source signal level may be a subband level of the source signal, and the displaying may be a step of partially changing a shape of an image of the source signal.

  According to the present invention, in the displaying step, the size of an image of a source signal other than the source signal having the highest level at a specific time is zero.

  According to the present invention, the image can be matched with metadata corresponding to the mix signal.

  According to still another aspect of the present invention, an input unit for inputting a playback command for a mix signal; the shape of the image of the source signal changes with time as the level of the source signal corresponding to the mix signal changes with time A display unit for displaying an image corresponding to the source signal, and the shape of the image is one of size, hue, brightness, and position in the screen. A mixed signal interface display apparatus including the above is provided.

  The present invention includes a step in which a remix information providing server coupled to a user terminal through a communication network receives a request for remix information from a user terminal; and a step of transmitting the requested remix information to the user terminal. The information represents a mix signal, a source signal included in the mix signal, a mix parameter for controlling the mix signal for each source signal, an upmix parameter for upmixing the mix signal, and the source signal. There is provided a signal processing method including at least one of an icon or an emoticon.

  The remix information includes a modified mix signal in which a part of the source signal is excluded from the source signal included in the mix signal.

  When receiving a request for transmission of the modified mix signal from the user terminal, the method includes a step of selecting a source signal to be excluded from the mix signal, and a modified mix from which the selected source signal is excluded. Generating a signal, wherein the transmitting step transmits the modified mix signal to the user terminal.

  The transmitting step includes transmitting the remix information to the user terminal when payment for the remix information is completed.

  The remix information providing server is connected to a financial institution or a mobile communication carrier through a communication network, and the settlement is a step of requesting user terminal authentication information from the user terminal, and the authentication information input from the user terminal. Requesting authentication by transmitting to the financial institution or the mobile telecommunications carrier, and transmitting settlement information to the financial institution or the mobile telecommunications carrier and requesting settlement when the user terminal authentication is completed. And including.

  The present invention relates to a signal processing method performed by a user terminal connected to a remix information providing server through a communication network, the step of requesting remix information by connecting to the server, and the settlement information requested from the server to the server. And receiving the remix information from the server when payment is completed, wherein the remix information includes a mix signal, a source signal included in the mix signal, and the mix signal. A signal processing method comprising: at least one of a mix parameter for controlling each source signal, an upmix parameter for upmixing the mix signal, an icon or an emoticon for expressing the source signal I will provide a.

  The present invention is a remix information providing server coupled to a user terminal through a communication network, wherein a remix information is requested from the user terminal and the requested remix information is transmitted to the user terminal, and the user terminal In response to the request, the database unit that extracts the requested remix information, the payment processing unit that processes the payment for the use of the remix information when the remix information use is charged, and the user terminal requested A control unit configured to transmit remix information to the user terminal, wherein the remix information controls a mix signal, a source signal included in the mix signal, and the mix signal for each source signal. Up the mix parameters for upmixing the mix signal Tsu box parameters, providing remix information providing server, which comprises at least one icon or emoticon for representing the source signal.

  The present invention is a user terminal that is coupled to a remix information providing server through a communication network, and requests a remix information from the remix information providing server and receives the requested remix information, and a user that receives control information The interface, the remix rendering unit that remixes the source signal included in the mix signal using the remix information and the control information, and the user terminal is controlled in general, and the mix signal is included in the mix signal. A control unit configured to remix the source signal, and the remix information includes a mix signal, a source signal included in the mix signal, a mix parameter for controlling the mix signal for each source signal, Upmix for upmixing mixed signals Hex parameter provides a user terminal, characterized in that it comprises at least one icon or emoticon for representing the source signal.

  The foregoing general description and the following detailed description are exemplary and are intended to further explain the scope of the claims.

  According to an aspect of the present invention, the volume level can be adjusted or the ambience (relative position) can be adjusted for each source signal included in a mix signal such as a video signal and an audio signal. The signal can be remixed very easily.

  According to another aspect of the present invention, it is possible to mute only the voice of a person or adjust only the volume level of the voice from the source included in the mix signal, so that the input is performed while the song file is played back. Using the device (remote controller), the user can very easily mute only the voice of the singer and listen only to the accompaniment music.

  According to still another aspect of the present invention, after a user frequently remixes a source (e.g., piano) as a shortcut key in a specific key (e.g., number key 1), the shortcut key is used. When adjusting the level adjustment or ambience for each source, the user can perform remixing very conveniently using an input device (remote controller) set according to his / her preference.

  According to yet another aspect of the invention, information about the source signal (or object signal) associated with the mix signal (or composite object signal) is continuously displayed on the screen so that the user can play the mix signal. During remixing and remixing, you can see what sources are included in the mix signal.

  According to yet another aspect of the present invention, while the mix signal is played back or remixed, the user can always check what sources (such as audio signals by instrument) are included for each mix signal, It is not necessary to perform a procedure to check what source is included in the corresponding mix signal before adjusting the level or ambience for each source.

  According to yet another aspect of the invention, information about the source signal (or object signal) associated with the mix signal (or composite object signal) is continuously displayed on the screen so that the user can play the mix signal. You can see what sources are included in the mix signal while performing or remixing.

  According to yet another aspect of the present invention, while the mix signal is played back or remixed, the user can always check what sources (such as audio signals by instrument) are included for each mix signal, There is no need to check the source included in the corresponding mix signal before adjusting the level or ambience for each source.

  In addition, the signal processing method according to the present invention allows a user to receive and use remix information from a remix information providing server.

  In addition, the signal processing method and apparatus according to the present invention can receive and use a modified mix signal in which a specific source signal is excluded from the mix signal.

  Also, the signal processing method according to the present invention allows the user to generate or transform remix information and upload it to the remix information providing server.

1 is a block diagram of a first remix signal encoding apparatus according to an embodiment of the present invention. FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when a stereo signal is used. 4 is a graph of a domain for processing a mix signal according to an embodiment of the present invention. FIG. 5 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention. FIG. 3 is a block diagram of a first remix signal decoding apparatus according to an embodiment of the present invention. FIG. 6 is a detailed diagram of the first remix signal decoding apparatus of FIG. 5 when a stereo signal is used. FIG. 5 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention. FIG. 5 is a block diagram illustrating a combination of a normal encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention. FIG. 10 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention used in combination with a conventional decoding apparatus. FIG. 4 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention. It is a block diagram of the input device by one Example of this invention. 1 is an implementation example of an input device according to an embodiment of the present invention; It is an example of the screen displayed after a 1st input part (source menu key) is input. It is an example of the screen displayed after a 2nd input part (audio | voice mute key) is input. It is an example of the screen displayed after a 3rd input part (voice adjustment key) is input. It is an example of the screen displayed after a 4th input part (ambience adjustment key) is input. It is an example of the screen displayed after the 1st source key (piano key) is input among the 5th input parts. It is an example of the screen displayed after a 2nd source key (violin key) is input among 5th input parts. It is a block diagram of the input device by the other Example of this invention. 1 is a diagram illustrating a configuration of a first remix signal decoding apparatus with which an input apparatus according to an embodiment of the present invention is associated; FIG. FIG. 6 is a diagram illustrating a configuration of a second remix signal decoding apparatus and a second interface apparatus with which input devices according to another embodiment of the present invention are associated with each other. 4 is a flowchart of an input method according to an embodiment of the present invention. It is an example of the screen displayed after a shortcut key registration command is input. 1 is a configuration diagram of an interface display device according to an embodiment of the present invention. 1 is an implementation example of an interface display device according to an embodiment of the present invention; 1 is an implementation example of an interface display device according to an embodiment of the present invention; It is an example of the screen displayed on the 1st display and the 2nd display when a specific source is selected. It is another example of the screen displayed on the 1st display and the 2nd display when a specific source is selected. It is a further another example of the screen displayed on the 1st display and the 2nd display when a specific source is selected. It is a further another example of the screen displayed on the 1st display and the 2nd display when a specific source is selected. It is a block diagram of the interface display apparatus by the other Example of this invention. 4 is a flowchart of an interface display method according to an embodiment of the present invention. 1 is a configuration diagram of an interface display device according to an embodiment of the present invention. 1 is an implementation example of an interface display device according to an embodiment of the present invention; 1 is an implementation example of an interface display device according to an embodiment of the present invention; 1 is an implementation example of an interface display device according to an embodiment of the present invention; 1 is an implementation example of an interface display device according to an embodiment of the present invention; It is a block diagram of the interface display apparatus by the other Example of this invention. 4 is a flowchart of an interface display method according to an embodiment of the present invention. 4 is a flowchart of an interface display method according to an embodiment of the present invention. It is an example of the screen where the icon classified by source signal of a specific mix signal was displayed. An icon for each source signal of a specific mix signal is an example of a screen expressing metadata. It is an example of a level adjustment screen. It is an example of the screen where the shape of the icon was changed according to the adjusted level. It is an example of an equalizer screen (level adjustment screen for each subband). It is an example of an ambience adjustment screen (relative position adjustment screen). It is a block diagram of the interface display apparatus by the further another Example of this invention. It is a block diagram of the interface display apparatus by the further another Example of this invention. 6 is a flowchart of an interface display method according to another embodiment of the present invention. It is an example of the screen from which the magnitude | size of an image changes in proportion to the level of a source signal. It is an example of the screen from which the brightness of an image changes in proportion to the level of a source signal. It is an example of a screen on which only an image having the highest level among source signals is displayed. It is an example of a screen on which only an image having the highest level among source signals is displayed. It is a figure which shows the remix information provision system by the Example of this invention. It is a block diagram of a remix information providing server according to an embodiment of the present invention. It is a module block diagram of the remix information provision server by the Example of this invention. 4 is a flowchart illustrating a method for providing remix information by a remix information providing server according to an embodiment of the present invention. 5 is a flowchart illustrating a method for receiving remix information by a user terminal according to an embodiment of the present invention. 5 is a flowchart illustrating a method for a user to transmit remix information to a remix information providing server according to an embodiment of the present invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  In particular, the term 'input unit' or 'key' in this specification should be understood as a means for inputting a specific input signal, and is not limited to an input method such as a button key or a touch sensor.

  In particular, the mix signal in the present invention includes an audio signal, a video signal, and the like, and should be understood as a concept including the composite object signal in this specification.

  The present invention provides an algorithm that can process a mix signal in units of an object signal. When the mix signal is processed in units of source signals, a great variety of effects can be generated. For example, when watching a movie, it is possible to increase only the volume of background music while keeping the volume of actors' voices constant. The source signal referred to in the present invention includes one or more sources (for example, a piano) constituting the mix signal. Source signal unit processing refers to characteristics (for example, localization, gain) associated with a specific source signal constituting the mix signal “individually ( individually) ”means that it can be modified. This “individually” means that modifying a characteristic associated with a particular source has only a minor effect that does not affect the characteristics of other source signals or is difficult to perceive. Means. For convenience of explanation, a stereo signal will be described below as an example, but the present invention is not limited to a stereo signal. In the present invention, a small amount of side information is transmitted in addition to the normal mix signal format (for example, PCM, MP3, MPEG-AAC). A remix signal can be generated using these mix signals and additional information.

  FIG. 1 is a block diagram of a first remix signal encoder according to an embodiment of the present invention. The first remix signal encoder includes an additional information generation unit 103 and an additional information encoding unit 105.

  Referring to FIG. 1, the additional information generation unit 103 generates additional information 104 using a normal mix signal 101 and a source signal 102 constituting the mix signal. The mix signal 101 can be mono, stereo, and multi-channel audio signal. The source signal 102 can be part or all of the source signals that make up the mix signal 101. The additional information 104 is information used to process a mix signal in units of source signals. The additional information 104 includes a mix parameter for remixing the mix signal. This mix parameter includes an encoder mix parameter (Encoder mix parameter) generated using the source signal in the encoder, and may include a blind mix parameter (Blind mix parameter) generated selectively using only the mix signal. it can. Examples of these mix parameters include a gain value and subband power for each source signal. A specific definition and generation method for the additional information 104 will be described with reference to FIG. The present invention also includes a method for generating the additional information 104 using only the source signal 102 constituting the mix signal. The additional information encoding unit 105 encodes the generated additional information 104 to generate an encoded additional information signal 106. The mix signal 101 and the additional information signal 106 are transmitted to the decoding device.

  FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when the mix signal is a stereo signal. As described above, the mix signal used in the present invention can be a mono, stereo, and multi-channel audio signal, but will be described with reference to the stereo signal 201 for convenience.

  An object of the present invention is to modify a stereo signal including these source signals so that M (0 ≦ M ≦ I) source signals are remixed. These source signals can be remixed into a stereo signal with different gain factors. The remix signal can be expressed as Equation 2 below.

Here, C _i and d _i are new gain factors for the M source signals to be remixed. These C _i and d _i can be provided at the decoder end. In this case, the additional information generation unit 206 can generate the additional information 207 using the stereo signal 201 and the M source signals 202.

As described above, an object of the present invention is to remix a stereo signal in units of source signals when a normal stereo signal and some additional information are given. As in the present invention, the remix signal expressed by Equation 2 cannot be completely generated from the mix signal expressed by Equation 1 using a very small amount of additional information.

Referring to FIG. 2, a normal stereo signal 201 and M source signals 202 included in the stereo signal 201 are input to the first remix signal encoding apparatus. The stereo signal 201 is delayed to some extent in order to synchronize with the additional information and can be used directly as an output signal. In order to generate additional information, the stereo signal 201 and the source signal 202 are decomposed through the filter bank 203 into subband-specific signals 204 and 205 in the time-frequency domain. That is, the stereo signal 201 and the source signal are processed in the time-frequency domain. This time-frequency domain will be described later. The subband-specific signal 204 is similarly processed at the center frequency of each subband. The subband pair 204 of the stereo signal 201 at a specific frequency is denoted by x ₁ (k) and x ₂ (k). Here, k is a time index of the subband signal. Similarly, the subband signals 205 of the M source signals 202 are denoted by S ₁ (k), S ₂ (k),..., S _M (k). No subband (frequency) index was used for clarity.

また、付加情報生成部２０６は、ステレオ信号２０１のサブバンド対２０４を用いて、サブバンド別にゲインファクタa_i及びb_iを生成する。ゲインファクタ a_i及びb_iは外部から直接与えられることができる。サブバンド別ショート−タイムサブバンドパワー及びゲインファクタを用いてサブバンド別付加情報２０７が生成される。付加情報生成部２０６は、ショート−タイムサブバンドパワー及びゲインファクタの他に、ステレオ信号に関連している他の情報を付加情報２０７として生成することができる。付加情報エンコーディング部２０８は、サブバンド別付加情報２０７を用いて、符号化された付加情報信号２０９を生成する。

Further, the additional information generation unit 206 generates gain factors a _i and b _i for each subband using the subband pair 204 of the stereo signal 201. The gain factors a _i and b _i can be given directly from the outside. Sub-band-specific additional information 207 is generated using the sub-band short-time sub-band power and the gain factor. The additional information generating unit 206 can generate other information related to the stereo signal as the additional information 207 in addition to the short-time subband power and the gain factor. The additional information encoding unit 208 generates an encoded additional information signal 209 using the subband-specific additional information 207.

For a plurality of stereo signals 201, gain factors a _i and b _i may be fixed. If a _i and b _i are variable with time k, these gain factors are generated as a function of time. Instead of directly quantizing and encoding these gain factors, they can be converted to other values that are more compatible with quantization and encoding.

Here, f _s represents the subband sampling frequency. For example, T = 40 ms can be used.

Similarly, b _i is calculated as in Equation 8 below.

  FIG. 3 shows a domain for processing an audio signal according to an embodiment of the present invention. As described above, the audio signal and the additional information are processed as subband-specific signals in the time-frequency domain, as shown in FIG. This time-frequency domain subband signal is perceptually derived. For example, a signal for each subband can be generated using an STFT (Short Time Fourier Transform) having a sine analysis and synthesis window having a length of about 20 ms. At this time, the STFT coefficients can be grouped so that one group has a bandwidth that is approximately twice the ERB (equivalent rectangular bandwidth).

  FIG. 4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention.

  Referring to FIG. 4, the downmixing unit 406 adds a plurality of source signals 401 to generate one sum signal. Unlike the first remix signal encoding device, the second remix signal encoding device transmits a sum signal 404 instead of transmitting a stereo signal. The additional information generation unit 403 generates additional information 405 using the source signal 401. The additional information 405 includes subband power and gain factor corresponding to each source signal. Further, the additional information 405 can include a parameter corresponding to the delay in the remix rendering unit. Similar to the first remix signal encoding apparatus, the additional information 405 can be transmitted after being converted into another value more suitable for quantization and encoding. The additional information encoding unit generates an encoded additional information signal 407 using the generated additional information 405. The generated sum signal 404 and additional information signal 407 are transmitted to the decoding apparatus. The present invention also includes an encoding apparatus that does not have the downmixing unit 402. In this case, the source signal 401 is not converted into the sum signal 404, and each source signal 401 is directly transmitted.

  FIG. 5 is a block diagram of a first remix signal decoding apparatus according to an embodiment of the present invention. The first remix signal decoding apparatus includes an additional information decoding unit 503 and a remix rendering unit 505.

  Referring to FIG. 5, the mix signal 501 and the additional information signal 502 are input to the first remix signal decoding apparatus. This mix signal 501 can be a mono, stereo or multi-channel audio signal. The additional information decoding unit 503 generates the additional information 504 by decoding the additional information signal 502. The additional information 504 includes a source signal gain factor and subband power included in the transmitted audio signal 501. A user-mix parameter 506 generated using control information directly provided by the user can be input to the remix rendering unit 505. The remix rendering unit 505 generates a remix signal 507 using the mix signal 501, the transmitted additional information 504, and the user mix parameter 506. A specific description regarding the method of generating the remix signal will be described later with reference to FIG. The remix signal 507 is generated as an Eq-channel mix signal (Eq-channel mix signal) having the same number of channels as that of the transmitted mix signal, or up with a larger number of channels than that of the mix signal. It can be generated as an up-channel mix signal.

  FIG. 6 is a detailed diagram of the first remix signal decoding apparatus of FIG. 5 when a stereo signal is used. As described above, the transmitted mix signal can be a mono, stereo, and multi-channel audio signal. For convenience, the stereo signal 601 will be described below as a reference.

Referring to FIG. 6, the stereo signal 601 is decomposed into a time-frequency domain subband-specific signal 604 through a filter bank 603. As shown in FIG. 6, the subband-specific signal 604 at a specific frequency is expressed by x ₁ (k) and x ₂ (k). The additional information decoding unit 605 decodes the transmitted additional information signal 602 to generate subband-specific additional information 606. Also, the user mix parameter 608 generated using the control information provided by the user can be input to the remix rendering unit 607. This user mix parameter 608 can be provided for each subband.

A method of generating the remix signal 609 generated by the remix rendering unit 607 is as follows.

  The subband-specific remix signal 609 can be expressed as Equation 10 below.

In order to generate the remix signal 609, least squares estimation can be used. If subband mix signal 604 x ₁ (k) and x ₂ (k) is given, the sub-band-specific remix signals 609 having different gains from each other as shown in Equation 11 below, linear subband mix signal 604 It can be estimated as a combination.

Here, w ₁₁ (k), w ₁₂ (k), w ₂₁ (k), and w ₂₂ (k) are weighting factors. At this time, the generated estimation error (estimation error) can be defined as Equation 12 below.

Similarly, w ₂₁ and w ₂₂ can be generated as in Equation 15 below.

  When the phases of the mix signal 604 are coherent with each other or almost synchronized, the value expressed as Equation 17 below approaches 1.

  At this time, the weight value can be expressed as shown in Equation 18 below.

The subband-specific remix signal 609 thus generated is converted into a time-domain remix signal 611 through the inverse filter bank 610 as described above. The remix signal 611 sounds like a remix signal generated by independently remixing each source signal using user mix parameters c _i and d _i generated using control information provided by the user.

  In the above, the description has been made centering on the remixing of a two-channel stereo signal. However, as described above, the present invention is not limited to a stereo signal, and can be extended to remixing a multi-channel audio signal, for example, a 5.1 channel audio signal. One skilled in the art can remix multi-channel audio signals, similar to the stereo signals described herein. In this case, Equation 11 can be expressed as Equation 19 below.

  Alternatively, certain channels of the mix signal channels can be left without being remixed. For example, for a 5.1 surround channel, remixing can be applied only to the front channel without modifying the two back channels. In this case, a 2 or 3 channel remixing algorithm is applied to the front channel.

  FIG. 7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention. The second remix signal decoding apparatus includes an additional information decoding unit 703, a spatial information integration unit 705, and a remix rendering unit 707.

  Referring to FIG. 7, a source signal sum signal 701 and an additional information signal 702 are input to a second remix signal decoding apparatus. The additional information decoding unit 703 decodes the additional information signal 702 to generate additional information 704. The additional information 704 includes a gain factor, a delay constant, a subband power, and the like. The additional information integration unit 705 uses the additional information 704 to separate the sum signal 701 into a plurality of source signals 706. The remix rendering unit 707 can generate the remix signal 709 using the source signal 706. At this time, the remix rendering unit 707 can generate the remix signal 709 using the mix parameter transmitted as the additional information. Further, the remix rendering unit 707 can selectively generate a remix signal 709 using a user mix parameter 708 generated using control information provided by the user.

  FIG. 8 is a block diagram illustrating a combination of a normal encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention. The mix signal 801 can be encoded by a normal encoding device 803 and converted into an encoded mix signal 805. The mix signal 801 can be a channel-specific signal or a source signal. The normal encoding device 803 includes not only a conventional encoding device such as an AAC or MP3 encoder but also an encoding device to be developed in the future. The remix signal encoding apparatus 804 according to the present invention generates the additional information signal 806 using the mix signal 801 and the source signal 802 included in the mix signal. The multiplexing unit 807 generates a bit stream 808 using the encoded mix signal 805 and additional information signal 806. As described above, the additional information signal 806 can be inserted into the auxiliary data area in the conventional mix signal format so as to be compatible with the conventional apparatus.

  FIG. 9 is a block diagram illustrating a connection between a normal decoding apparatus and a signal processing apparatus according to an embodiment of the present invention. The demultiplexer 902 separates the encoded mix signal 903 and the additional information signal 904 from the transmitted bit stream 901. Thereafter, the normal decoding device 905 decodes the encoded mix signal 903 to generate a mix signal 906 that can be used in the remix signal decoding device 907 according to the present invention. The normal decoding device 905 includes an encoding device to be developed in the future in addition to a conventional encoding device such as an AAC or MP3 decoder. The mix signal 906 can be a channel-specific signal or a source signal. The remix signal decoding apparatus 907 according to the present invention can convert the mix signal 906 into the remix signal 908 using at least one of the additional information signal 904 and the user mix parameter 909.

  FIG. 10 is a block diagram of a fifth signal processing apparatus according to an embodiment of the present invention. Referring to FIG. 10, the fifth signal processing apparatus includes a mix signal decoding unit 1001, a parameter generation unit 1002, and a remix rendering unit 1008. Optionally, an effector 1011 can be included. The parameter generation unit 1002 may include a blind mix parameter generation unit 1003, a user mix parameter generation unit 1004, and a remix parameter generation unit 1005. The remix parameter generation unit 1005 includes an eq-mix parameter generation unit 1006, and can optionally include an upmix parameter generation unit 1007. Further, the remix rendering unit 1008 includes an eq-mix rendering unit 1009, and may optionally include an upmix rendering unit 1010.

  The mix signal decoding unit 1001 generates a mix signal by decoding the encoded mix signal transmitted from the encoding end. The parameter generation unit 1002 receives additional information and user control information or configuration information transmitted from the encoding end. User control information can be generated at the decoder end instead of being transmitted from the encoder end. The user mix parameter generation unit 1004 generates user mix parameters using the user control information. The additional information transmitted from the encoder end can include an encoder mix parameter. Also, the blind mix parameter generation unit 1003 can generate a blind mix parameter (Blind-Mix Parameter) using the mix signal. The encoder mix parameter and the blind mix parameter are alternatively input to the remix parameter generation unit 1005.

  The remix parameter generation unit 1005 generates a remix parameter using the additional information and the user mix parameter. This remix parameter can be generated to be applicable to the channel of the remix signal. The eq-mix parameter generation unit 1006 included in the remix parameter generation unit 1005 generates a remix parameter used to generate a remix signal having the same number of channels as the number of channels of the mix signal, and includes the remix parameter generation unit 1005. The upmix parameter generation unit 1007 that can generate a remix parameter used to generate a remix signal having a larger number of channels than the number of channels of the mix signal. These remix parameters are input to the remix rendering unit 1008.

  The eq-mix rendering unit 1009 included in the remix rendering unit 1008 generates an eq-channel remix signal (Eq-channel remix signal) having the same number of channels as the number of channels of the mix signal using the remix parameter and the mix signal. . The upmix rendering unit 1010 that can be included in the remix rendering unit 1008 uses the remix parameter and the mix signal generated by the upmix parameter generation unit 1007 to use an up-channel remix having a larger number of channels than the number of channels of the mix signal. A signal (Up-channel remix signal) is generated. The upmix rendering unit 1010 can generate an upchannel remix signal using the remix signal generated by the eq-channel rendering unit 1009.

  Therefore, the fifth signal processing apparatus can output the mix signal transmitted from the encoding end as it is, output it as an eq-channel remix signal, or output it as an up-channel remix signal. Optionally, the remix rendering unit can give various effects to the remix signal using information provided from the effector 1011.

  FIG. 11 is a diagram illustrating a configuration of an input device according to an embodiment of the present invention, and FIG. 12 is a diagram illustrating an implementation example of the input device according to an embodiment of the present invention. Referring to FIGS. 11 and 12, an input device 1000 according to an embodiment of the present invention (hereinafter abbreviated as 'input device 1000') is associated with a first remix signal decoding device 500, and includes a first input. Section 1110 (source menu key), second input section 1120 (audio mute key), third input section 1130 (audio adjustment key), fourth input section 1140 (ambience adjustment key), fifth input section 1150 (first source) A key 1150a (piano key), a second source key 1150b (violin key), a direction movement key 1160a, a pointing device 1160b, a numeric key 1170, a storage unit 1200, a control unit 1300, and a display 1400 are included.

  First, since the first remix signal decoding apparatus 500 has the same configuration as that of the first remix signal decoding apparatus described with reference to FIG. 5, a detailed description of the components is omitted. Below, each component of the input device 1000 is demonstrated concretely.

  The first input unit 1110 (source menu key) is an input device that converts the input signal into a menu display command for each source by the control unit 1300. The menu for each source is a screen on which a list of sources included in the specific mix signal is displayed, and the volume level and the like can be adjusted for each source while the menu for each source is displayed. An example of a screen (source-specific menu display screen) displayed after the first input unit (source menu key) is input is shown in FIG. Here, the menu display screen for each source can be displayed on the display 1400 of the first interface device 550. The first input unit 1110 (source menu key) may be implemented by a toggle type button key or a touch sensor in which command execution and cancellation are repeated according to the number of times the button is pressed, but the present invention is not limited to this.

  The second input unit 1120 (voice mute key) is an input device that converts the input signal into a voice mute command by the control unit 1300. The voice mute command is a command for causing the level of a human voice (voice) to become zero in the source included in the specific mix signal. FIG. 14 shows an example of a screen displayed after the second input unit (audio mute key) is input. Similarly, the second input unit 1120 (voice mute key) can be implemented by a toggle type button key or a touch sensor in which command execution and cancellation are repeated according to the number of times of pressing, but the present invention is not limited to this.

  The third input unit 1130 (voice adjustment key) is an input device in which the input signal is converted into a voice level adjustment command by the control unit 1300. The voice level adjustment command is a command for raising or lowering only the level of a human voice among the sources included in the specific mix signal. FIG. 15 shows an example of a screen displayed after the third input unit (voice adjustment key) is input. The second input unit 1120 (voice mute key) is also for inputting a level up command and a level down command, and includes two button keys, a jog shuttle, two-way jog buttons, two-way touch scroll, Although it can be embodied by a single touch sensor, a touch wheel or the like, the present invention is not limited to this.

  The fourth input unit 1140 (ambience adjustment key) is an input device in which the input signal is converted into an ambience screen display command by the control unit 1300. The ambience screen is a screen on which the relative position of the source included in the specific mix signal is displayed, and the position of each source can be changed using the direction movement key 1160a or the pointing device 1160b. FIG. 16 shows an example of a screen (ambience screen) that is displayed after the fourth input unit (ambience adjustment key) is input. Similarly, the fourth input unit 1140 (voice mute key) can be implemented by a toggle type button key or a touch sensor in which command execution and cancellation are repeated according to the number of times of pressing, but the present invention is not limited to this.

  The fifth input unit 1150 is an input device that converts the input signal into a source sound mute command by the control unit 1300. The source sound mute command is a command for preventing the source signal corresponding to the specific instrument from being reproduced. The fifth input unit 1150 may be, for example, a first source key 1150a (piano key) for setting the level of the source signal corresponding to “piano” to approximately 0, and the source signal corresponding to “violin”. A second source key 1150b (violin key) for making the level substantially zero may be used. FIG. 17 shows an example of a screen displayed after the first source key (piano key) is input in the fifth input section, and the second source key (violin key) is input in the fifth input section. FIG. 18 shows an example of a screen that is displayed after being performed. Each of the first source key and the second source key of the fifth input unit 1150 can be implemented by a toggle type button key or a touch sensor in which command execution and cancellation are repeated according to the number of times the button is pressed. It is not limited to.

  The direction movement key 1160a is an input device in which the input signal is converted into a command for direction movement by the control unit 1300. The command regarding the direction movement can be a specific source level adjustment command when the source-specific menu screen is displayed, and can be a specific source position adjustment command when the ambience screen is displayed. The direction movement key 1160a can be implemented with four button keys, four touch sensors, four-direction touch scroll, a touch wheel, and the like, but the present invention is not limited to this.

  The pointing device 1160b is an input device in which the input signal is converted into a command for position designation by the control unit 1300. As with the direction movement key 1160a, the command related to the position designation can be a specific source level adjustment command when the source-specific menu screen is displayed, and can be a specific source position adjustment command when the ambience screen is displayed. Can be. The pointing device 1160b can be implemented by a mouse, a trackball, a touch pad, a touch screen, a touch panel (and a stylus pen), but the present invention is not limited to this.

  The numeric key 1170 is an input device whose input signal is converted into a command related to numeric input by the control unit 1300. The command related to the number input can be a command for selecting a specific source that matches the number in a state where the menu screen by source or the ambience screen is displayed. For example, when “1. voice, 2. piano, 3. violin, 4. flute” is displayed on the menu screen by source, when the user selects “No. 3 key” from among the numeric keys 1170, No. 3 The 'violin' that matches is selected. The numeric key 1170 can be implemented with ten button keys, ten touch sensors, a touch panel, and the like, but the present invention is not limited to this.

  The storage unit 1200 stores command information for each key. The command information for each key includes each input unit (first input unit 1110, second input unit 1120, etc.) (including numeric keys 1170) and corresponding commands (source-specific menu display command, voice mute command, etc.). means. Meanwhile, command information for each key can be newly stored or changed through a shortcut key registration process by the user, and a specific description of the shortcut key registration process will be described later with reference to FIG. On the other hand, the storage unit 1200 further stores data related to a source-specific menu screen, an audio mute screen, an audio adjustment screen, an ambience adjustment screen, and the like.

  The control unit 1300 converts the input signal received from the input unit (1110, 1120, 1130, 1140, 1150, etc.) into each command based on each key-specific command information stored in the storage unit 1200. Specifically, the input signal received through the first input unit 1110 is converted into a menu display command for each source, the input signal received through the second input unit 1120 is converted into an audio mute command, and then converted into an audio mute command. Based on this, the control information 506 relating to the mix signal is output, the input signal received through the third input unit 1130 is converted into an audio level adjustment command, and the input signal received through the fourth input unit 1140 is converted into an ambience screen display command. The input signal received through the fifth input unit 1150 is converted into a source sound mute command.

  In addition, the control unit 1300 displays an interface screen based on the source information si received from the first remix signal decoding apparatus 500 by an input of the input unit (1110, 1140, etc.). For example, when a source menu display command is input through the first input unit 1110, a source menu is displayed on the display 1400 based on the source information si, and an ambience screen display command is input through the fourth input unit 1140. The ambience screen is displayed based on the source signal information si corresponding to the mix signal.

  Also, the control unit 1300 outputs control information 506 related to the mix signal based on a command through the input unit (1120, 1130, 1140, etc.), and inputs the control information 506 to the first remix signal decoding apparatus 500. Here, the control information 506 has already been described with reference to FIG. Specifically, when the source-specific menu screen is displayed on the display, the control unit 1300 converts the input signal received through the pointing device 1160b or the direction movement key 1160a into a source-specific level adjustment command, When the ambience screen is displayed on the display, the input signal received through the pointing device 1160b or the direction movement key 1160a is converted into an ambience adjustment command. In the process in which the control unit 1300 converts these input signals into corresponding commands, the command information for each key stored in the storage unit 1200 can be referred to. Control information 506 related to the mix signal can be output based on the converted instructions (level adjustment instruction for each source, ambience adjustment instruction, etc.).

  The display 1400 is a display device that displays a menu screen by source or an ambience screen under the control of the control unit 1300. Furthermore, it is possible to display a screen (FIGS. 14, 15, 17, 18, etc.) displayed after each input unit (first input unit 1110 or the like) is input.

  FIG. 19 is a diagram showing a configuration of an input device according to another embodiment of the present invention. Referring to FIG. 19, an input device 2000 (hereinafter abbreviated as “input device 2000”) according to another embodiment of the present invention is associated with the first remix signal decoding device 500 and the first interface device 550. 1 input unit 2110 (source menu key), 2nd input unit 2120 (audio mute key), 3rd input unit 2130 (audio adjustment key), 4th input unit 2140 (ambience adjustment key), 5th input unit 2150 (first) 1 source key 2150a (piano key), second source key 2150b (violin key), direction movement key 2160a, pointing device 2160b, number key 2170, storage unit 2200, control unit 2300, display 2400, and wireless communication unit 2500. Of these components, components other than the control unit 2300 and the wireless communication unit 2500 are necessary. Since a substantially same components of the same names of the components of an embodiment of the present invention described in conjunction with FIGS. 11 and 12 above, a description thereof will be omitted.

  In addition, since the first remix signal decoding apparatus 500 has the same configuration as the first remix signal decoding apparatus described with reference to FIG.

  Meanwhile, the first interface device 550 is a device that provides an interface from the first remix signal decoding device 500 to the input device 2000, and includes a wireless communication unit 552, a storage unit 554, an interface control unit 556, and a display unit 558.

  The wireless communication unit 552 of the first interface device 550 is a device for performing wireless communication with the input device 2000, and the wireless communication method here is not limited to the infrared communication method. The storage unit 554 stores data related to a source-specific menu screen, an audio mute screen, an audio adjustment screen, an ambience adjustment screen, and the like. The interface control unit 556 displays an interface screen based on the source information si received from the first remix signal decoding apparatus 500, and provides control information to the first remix signal decoding apparatus 500 based on a command received from the input apparatus 2000. Enter 506. Here, since the control information 506 has been described with reference to FIG. 5 above, a specific description thereof will be omitted. The display 558 is a display device that displays an interface screen under the control of the interface control unit 556. When the input device 2000 includes the display 2400, the display 558 of the first interface device 550 may be omitted.

  Below, control part 2300 and radio | wireless communication part 2500 are demonstrated among each component of the input device 2000. FIG.

  The control unit 2300 converts the input signal received through the first input unit 2110 into a menu display command by source, converts the input signal received through the second input unit 2120 into an audio mute command, and the third input unit 2130. The input signal received through the fourth input unit 2140 is converted into an audio level adjustment command, the input signal received through the fourth input unit 2140 is converted into an ambience screen display command, and the input signal received through the fifth input unit 2150 is muted as a source sound. Convert to instruction. In addition, when the menu screen for each source is displayed on the display, the control unit 2300 converts the input signal received through the pointing device 2160b or the direction movement key 2160a into a level adjustment command for each source, and the ambience screen is displayed. In the state displayed on the screen, the input signal received through the pointing device 2160b or the direction movement key 2160a is converted into an ambience adjustment command. In the process in which the control unit 2300 converts these input signals into corresponding commands, the command information for each key stored in the storage unit 2200 can be referred to.

  The wireless communication unit 2500 transmits the command converted by the control unit 2300 to an external device, in particular, the first interface device 550. In addition, when the first interface device 550 transmits data related to the source-specific menu screen or data related to the ambience screen, the wireless communication unit 2500 receives the data.

  FIG. 20 is a diagram illustrating a configuration of a second remix signal decoding apparatus and a second interface apparatus associated with an input apparatus according to an embodiment of the present invention. Referring to FIG. 20, the input device 1000 is associated with the second remix signal decoding device 700. Since the second remix signal decoding apparatus 700 has been described with reference to FIG. 7 above, a detailed description thereof will be omitted. The second interface device 750 receives the source information si from the second remix signal decoding device 700 instead of the first remix signal decoding device 500 and inputs the control information 708 to the second remix signal decoding device 700. The component elements 752, 754, 756, and 758 are substantially the same as the component elements 552, 554, 556, and 558 having the same names in the first interface device 550, and thus the description thereof is omitted.

  FIG. 21 is a diagram illustrating a configuration of a second remix signal decoding apparatus and a second interface apparatus with which an input apparatus according to another embodiment of the present invention is associated. Referring to FIG. 21, the input device 2000 is associated with the second remix signal decoding device 700 and the second interface device 750. Since the second remix signal decoding apparatus 700 has been described in detail with reference to FIG. 7, the detailed description thereof will be omitted. The second interface device 750 receives the source information si from the second remix signal decoding device 700 instead of the first remix signal decoding device 500 and inputs the control information 708 to the second remix signal decoding device 700. Since the components 752, 754, 756, and 758 are substantially the same as the components 552, 554, 556, and 558 having the same names in the first interface device 550, the description thereof is omitted.

  FIG. 22 is a flowchart illustrating an input method according to an embodiment of the present invention. Referring to FIG. 22, when the user selects shortcut key registration, the input apparatus receives a shortcut key registration command (S110). An example of a screen displayed after a shortcut key registration command is input is shown in FIG. Thereafter, the user selects a key (e.g., No. 1 key) to be used as a shortcut key from among a plurality of keys (e.g., numeric keys) provided in the input device (step S120). The input device displays a list of sources that can be used for the mix signal (S130). For example, the source that can be used for the mix signal can be an audio audio signal or an audio signal by instrument (speech, piano, violin, flute, etc.). Then, the user selects a source to be matched with the shortcut key by selecting one from the source list (step S140). For example, a piano can be selected. Thereafter, the user selects a function (eg, sound mute) to be used as a shortcut key from one or more functions (eg, sound mute, ambience adjustment) (step S150). When the user selects the key, source, and function in steps S120, S140, and S150 as described above, the input device stores the selected key, the selected source, and the selected function as shortcut key registration information ( Step S160). For example, a piano signal sound mute command is matched with the first key and stored. Then, when the first key is input, the input device converts the input signal of the first key into a sound mute command for the piano signal.

  FIG. 24 is a diagram showing a configuration of an interface display device according to an embodiment of the present invention. 25A and 25B are implementation examples of an interface display device according to an embodiment of the present invention. 24, 25A, and 25B, an interface display device 3000 according to an exemplary embodiment of the present invention is associated with the first remix signal decoding device 500, and includes an input unit 3100, a control unit 3200, a first unit. 1 display 3300a and 2nd display 3300b are included. Here, the first remix signal decoding apparatus 500 includes an additional information decoding unit 503 and a remix rendering unit 505. The first remix signal decoding apparatus 500 has substantially the same configuration and functions as those of the first remix signal decoding apparatus described with reference to FIG. Since they are the same, a detailed description thereof is omitted.

  The input unit 3100 is an input device for selecting one of a plurality of mix signals. Here, the mix signal may be the channel-specific audio signal 501 described with reference to FIG. Among the plurality of mix signals, the mix signal 501p selected by the input unit 3100 and the additional information signal 502p of the mix signal 501p are input to the first remix signal decoding apparatus 500. The input unit 3100 may also be used to select one of the sources included in the mix signal. Further, the input unit 3100 further includes a command for adjusting the level of the source signal, level adjustment for each subband of the source signal (equalizer adjustment). ) And an instruction related to ambience adjustment of the source signal may be input. The input unit 3100 may be implemented by a direction movement key, a touch wheel, or the like in addition to a pointing device such as a mouse, a trackball, or a touchpad, and may be implemented by a remote controller using wireless communication such as infrared communication. Of course.

  The control unit 3200 inquires the source signal information si corresponding to the mix signal 501p selected by the input unit 3100, and then configures a list of source signals (hereinafter, a source list) based on the source signal information si, Is displayed on the first display 3300a. Here, the source signal information si is information on the source signal included in the mix signal 501p selected by the input unit 3100, and may be information including the instrument name of each source signal. The source signal information si may be the information of the source signal 102 described with reference to FIG. On the other hand, the source signal information si can further include metadata (musical instrument player name, production date, etc.). Meanwhile, the source signal information si may be information received from the additional information decoding unit 503 of the first remix signal decoding apparatus 500 by the control unit 3200. Here, the source list may include one or more of the instrument name, singer name, and character name of the source signal 502p.

  The control unit 3200 also receives various commands related to remixing (such as a command related to level adjustment of the source signal, a command related to level adjustment of each subband of the source signal, and a command related to ambience adjustment of the source signal) from the input unit 3100. The command is converted into control information 506 and input to the remix rendering unit 505. Here, the control information 506 is the same as the control information 506 described with reference to FIG. 5 and the control information 608 described with reference to FIG.

  The first display 3300a is a display on which a list of sources is displayed under the control of the control unit 3200. The list of sources displayed on the first display 3300a is continuously displayed on the first display 3300a while the mix signal is selected. 'While the mix signal is selected' means, for example, that the selected mix signal is being reproduced or remixed (level adjustment, ambience adjustment, etc.). If another mix signal is selected by the user, a list of sources matched with the other mix signal is displayed on the first display 3300a. The first display 3300a may be implemented with an LCD or the like, and may be implemented with a matrix of a plurality of light emitting devices, but the present invention is not limited thereto. When the first display 3300a is implemented with a plurality of light emitting elements, a list of sources is displayed by blinking the plurality of light emitting elements.

  The second display 3300b is a display on which metadata, a volume level adjustment screen, an equalizer adjustment screen, an ambience adjustment screen, and the like are displayed under the control of the control unit 3200. 26 to 29 are examples of screens displayed on the first display and the second display when the specific source is selected. FIG. 26 is a table of “1... From the source list displayed on the first display 3300a. FIG. 27 shows an example in which the metadata of the selected source (1. audio) is displayed on the second display 3300b when “audio” is selected, and FIG. 27 shows the volume level adjustment of the selected source (1. audio) In this example, the screen is displayed on the second display 3300b. FIG. 28 shows an example in which the equalizer adjustment screen (level adjustment screen for each subband) of the selected source (3. violin) is displayed on the second display 3300b, and FIG. 29 shows the selected source (2. piano). ) Is an example in which the ambience adjustment screen is displayed on the second display 3300b. 26 to 29, it can be seen that the list of sources continues to be displayed on the first display 3300a even if the screen displayed on the second display 3300b changes.

  Meanwhile, the second display 3300b and the first display 3300a are not necessarily mechanically or device-separated, and thus the second display 3300b and the first display 3300a may be embodied as one panel. Of course.

  FIG. 30 is a diagram showing the configuration of an interface display device according to another embodiment of the present invention. Referring to FIG. 6, an interface display device 4000 according to another embodiment of the present invention includes an input unit 4100, a control unit 4200, a first display 4300a, and a second display 4300b, and is related to the second remix signal decoding device 700. I'm happy. Here, the first remix signal decoding apparatus 700 includes an additional information decoding unit 703, an additional information integration unit 705, and a remix rendering unit 707. The second remix signal decoding apparatus described with reference to FIG. Since the configuration and function are substantially the same, a detailed description thereof will be omitted.

  Meanwhile, the interface display device 4000 according to another embodiment of the present invention is different from the interface display device 3000 according to an embodiment of the present invention in that the second remix signal decoding device 700 is not the first remix signal decoding device 500. The components 4100, 4200, 4300a, and 4300b are different from the interface display device 3000 according to the embodiment of the present invention except that the composite object signal 701p is input instead of the mix signal 501p. Since the components 3100, 3200, 3300a, and 3300b having the same names are substantially the same, a description of the components is omitted.

  FIG. 31 is a flowchart illustrating an interface display method according to an embodiment of the present invention. The term mix signal used in the following should be understood as a concept that includes even the concept of a composite object signal. The interface display method according to the embodiment of the present invention may be performed by the interface display device 3000 according to the embodiment of the present invention described above, or may be performed by the interface display device 4000 according to another embodiment of the present invention. .

  Referring to FIG. 31, first, the interface display device displays information of one or more mix signals that can be stored or transmitted on the display as a list of mix signals (S210). In step S210, the user selects a specific mix signal to be reproduced or remixed from the displayed list of mix signals using the input device (step S220). Then, the interface display device extracts source signal information corresponding to the mix signal from the additional information corresponding to the mix signal selected in operation S220 (operation S230).

  The interface display device constructs a list of sources based on the source signal information extracted in operation S230 (operation S240). Thereafter, the interface display device displays a list of sources on the first display (S250).

  The user confirms a list of sources displayed on the first display, and selects one source from the list (S260). Then, the interface display device displays the source metadata (the volume level adjustment screen, the equalizer adjustment screen, the ambience adjustment screen, etc. depending on the setting) selected in step S260 on the second display (step S270).

  Thereafter, when the selection of the mix signal selected in step S220 is completed (for example, when the user ends reproduction of the corresponding mix signal or selects another mix signal) (“Yes” in step S280), S250. The display of the source list starting from the step is terminated (step S290).

  FIG. 32 is a diagram showing a configuration of an interface display device according to an embodiment of the present invention. Referring to the figure, an interface display device 5100 according to an embodiment of the present invention includes an input unit 5110, a storage unit 5120, a control unit 5130, and a display 5140, and is associated with the first remix signal decoding device 500. . Here, the first remix signal decoding apparatus 500 includes an additional information decoding unit 503 and a remixing rendering unit 506. Since it is substantially the same, the specific description is abbreviate | omitted.

  The input unit 5110 is an input device for selecting one from a plurality of mix signals. Here, the mix signal may be the mix signal (channel-specific audio signal) 501 described in conjunction with FIG. The mix signal 501p selected from the plurality of mix signals by the input unit 5110 and the additional information signal 502p of the mix signal 501p are input to the first remix signal decoding apparatus 500. The input unit 5110 may also have a function of inputting a command related to level adjustment of the source signal, a command related to level adjustment for each subband of the source signal, and a command related to ambience adjustment of the source signal. The input unit 5110 can be implemented by a direction movement key, a touch wheel, or the like in addition to a pointing device such as a mouse, a trackball, or a touch pad. You can also

  The storage unit 5120 is a storage device that stores icon-specific image data. Specifically, it can be stored together with the category and keyword corresponding to the icon. In addition to text, icons can be displayed as images using image data.

  The controller 5130 inquires of the source signal information si corresponding to the mix signal 501p selected by the input unit 5110, and then displays an icon corresponding to the source signal based on the source signal information si. Here, the source signal information si is information regarding the source signal included in the mix signal 501p selected by the input unit 5110, and may be information including the instrument name of each source signal. The source signal information si may be the information of the source signal 102 described with reference to FIG. On the other hand, the source signal information si may further include metadata (musical instrument player name, vocalist name, etc.). Meanwhile, the source signal information si can be received by the control unit 5130 from the additional information decoding unit 503 of the first remix signal decoding apparatus 500.

  On the other hand, the icon corresponding to the source signal may be displayed as text, but may be displayed as an image using image data stored in the storage unit 5120.

  The control unit 5130 also receives various commands related to remixing from the input unit 5110 (such as a command related to the level adjustment of the source signal, a command related to the level adjustment of each subband of the source signal, and a command related to the ambience adjustment of the source signal). The command is converted into control information 506 and input to the remixing rendering unit 505. Here, the control information 506 is the same as the control information 506 described with reference to FIG. 5 and the control information 608 described with reference to FIG.

  The display 5140 is a display device that displays an icon corresponding to the source signal under the control of the control unit 5130.

  33A to 33D are examples in which an interface display device according to an embodiment of the present invention is implemented. First, FIG. 33A is an example in which an interface display device according to an embodiment of the present invention is implemented as a mobile terminal integrally with the first remix signal decoding device 500. FIG. 33B is an example in which the input unit 5110 of the interface display device 5100 according to an embodiment of the present invention is implemented with a mouse and a touch pen connected to the first remix signal decoding device 500 by a wire, and the display 5140 is also input. Similarly to the unit 5110, the monitor is connected to the first remix signal decoding apparatus 500 by wire. 33D, the display 5140 of the interface display device 5100 according to an embodiment of the present invention is connected to the first remix signal decoding device 500 by wire, and the input device 5110 is connected to the first remix signal decoding device 500. It is an example embodied by a remote controller that performs wireless communication. Referring to FIG. 33D, the input unit 5110 of the interface display device 5100 according to an embodiment of the present invention is implemented by a remote controller as in FIG. 33C, and the display 5140 is provided in the remote controller. I understand. As described above, the interface display device 5100 and the first remix signal decoding device 500 according to an embodiment of the present invention may be variously implemented.

  FIG. 34 is a diagram showing the configuration of an interface display device according to another embodiment of the present invention. Referring to the figure, an interface display device 5200 according to another embodiment of the present invention includes an input unit 5210, a storage unit 5220, a control unit 5230, and a display 5240, and is associated with the second remix signal decoding device 700. Yes. Here, the first remix signal decoding apparatus 700 includes an additional information decoding unit 703, an additional information integration unit 705, and a remixing rendering unit 708. The second remix signal decoding described with reference to FIG. Since the configuration and functions of the apparatus are substantially the same, a detailed description thereof will be omitted.

  Meanwhile, the interface display device 5200 according to another embodiment of the present invention is compared with the interface display device 5100 according to an embodiment of the present invention, instead of the first remix signal decoding device 500, the second remix signal decoding device 700. The components 5210, 5220, 5230 and 5240 are different from each other only in that the composite object signal 701p is input instead of the mix signal 501p. Since the components 5110, 5120, 5130, and 5140 having the same names in 5100 are substantially the same, the description of the components is omitted.

  35A and 35B are flowcharts illustrating an interface display method according to an embodiment of the present invention. The term mix signal used in the following should be understood as a concept that includes even the concept of a composite object signal. The interface display method according to an embodiment of the present invention may be performed by the interface display device 5100 according to the embodiment of the present invention described above, or by the interface display device 5200 according to another embodiment of the present invention. May be.

  First, referring to FIG. 35A, the interface display device displays information on one or more mix signals that are already stored or can be transmitted on the display as a list of mix signals (S410). In step S420, the user selects a specific mix signal to be reproduced or to check source signal information from the displayed list of mix signals using the input device (step S420). Then, the interface display device extracts source signal information corresponding to the mix signal from the additional information corresponding to the mix signal selected in operation S420 (operation S430). If the source signal information extracted in operation S430 includes the metadata of the mix signal, the metadata is extracted from the source signal information (operation S440).

  Thereafter, the interface display apparatus inquires an icon corresponding to the source signal based on the source signal information extracted in operation S430 (operation S450). Here, the icon can be composed of text or an image. If metadata is extracted in operation S440, an image matched with the metadata can be queried. Thereafter, the icon inquired in step S450 is displayed on the display (step S460). FIG. 36A shows an example of a screen on which an icon for each source signal of the specific mix signal is displayed. FIG. 36A is a screen on which icons corresponding to piano, violin, vocalist (girls), and flute are displayed when the source signal information is “piano, violin, vocalist (girls), flute”. As described above, the interface display method according to the embodiment of the present invention displays the icon for the specific mix signal by performing steps S410 to S450. FIG. 36B shows an example of a screen on which the source signal icon of the specific mix signal represents metadata. FIG. 36B shows that when the metadata included in the source signal information is “piano performer: Jung Mun-Hun” and “singer: Cho Sumi”, the icon corresponding to the piano is Jung Mun-Hun. The screen is composed of photos, and the icon corresponding to vocal music is composed of photos of Cho Sumi. Hereinafter, a process after a specific icon is executed among icons displayed through such a process will be described.

  Referring to FIG. 35B, when the user selects one of the icons displayed in step S450 and inputs an execution command related to the specific icon, the interface display apparatus receives the execution command for the selected icon (step S510). ). The icon execution command, that is, the process performed when the icon is executed may vary depending on the setting. Three cases (level adjustment, equalizer adjustment, and ambience adjustment) will be described below.

  If the icon execution command is set to level adjustment (volume adjustment) (“Yes” in step S520), a level adjustment screen of the source signal corresponding to the icon executed in step S510 is displayed (step S522). ). An example of the level adjustment screen is shown in FIG. 37A. FIG. 37A is a screen on which a control bar that can adjust the flute level (volume) is displayed when the “flute” icon is executed out of a total of four icons. The user moves a control bar using a pointing device on the screen as shown in FIG. 37A or inputs a command related to level adjustment of a specific source signal (eg, flute) using a keypad direction key or the like. Can do. When such a command for level adjustment is received (“Yes” in step S524), the shape of the icon is changed according to the adjusted level (step S526). Here, the shape of the icon is a concept including one or more of size, hue, and brightness. An example of the screen in which the shape of the icon is changed according to the adjusted level is shown in FIG. 37B. Referring to FIG. 37B, it can be seen that the size of the icon corresponding to the flute increases as the flute level is adjusted higher. Thereafter, the level adjustment command input in step S524 is converted into control information, thereby remixing the mix signal selected in step S420 (step S550). Here, the control information is information input to the first remix signal decoding device or the second remix signal decoding device, and has been described above together with the interface display device according to an embodiment of the present invention, and thus description thereof is omitted. To do.

  If the icon execution instruction is set to equalizer adjustment (level adjustment for each subband) (“Yes” in step S530), the level adjustment screen of the source signal corresponding to the icon executed in step S510 is displayed. (Step S532). An example of the equalizer screen (level adjustment screen for each subband) is shown in FIG. FIG. 38 is a screen on which a control bar for each subband is displayed so that the piano equalizer can be adjusted when the “piano” icon is executed out of a total of four icons. On the screen as shown in FIG. 38, the user can input a command related to level adjustment for each subband of a specific source signal (for example, piano) by moving a control bar for each subband using a pointing device. When such a command for subband level adjustment is received (“Yes” in step S534), the shape of the icon is partially changed according to the adjusted subband level (step S536). For example, when the level of the low frequency band is adjusted high, the brightness of the portion corresponding to the left side of the icon can be reduced. Then, the mix signal selected in step S420 is remixed by converting the subband level adjustment command input in step S534 into control information (step S550).

  If the icon execution command is set to ambience adjustment (relative position adjustment) (“Yes” in step S540), the ambience adjustment screen (relative position) of the source signal corresponding to the icon executed in step S510. Adjustment screen) is displayed (step S542). An example of the ambience adjustment screen (relative position adjustment screen) is shown in FIG. FIG. 39 is a screen on which the relative position of the piano is displayed so that the ambience of the piano can be adjusted when the “piano” icon among the four icons in total is executed. The user can input an instruction related to ambience adjustment of a specific source signal (for example, piano) by moving an icon using a pointing device, a direction key, or the like on a screen as shown in FIG. When such an ambience adjustment command is received (“Yes” in step S544), the icon shape is changed according to the adjusted level for each subband (step S546). For example, when the relative position of the adjusted source signal approaches the position of the user (listener), the brightness of the corresponding icon can be increased. Thereafter, in step S544, the input ambience adjustment command is converted into control information, thereby remixing the mix signal selected in step S420 (step S550).

  FIG. 40 is a block diagram of an interface display device according to still another embodiment of the present invention. Referring to FIG. 40, an interface display device 6100 according to another embodiment of the present invention includes an input unit 6110, a storage unit 6120, a control unit 6130, a display 6140, and is associated with the first remix signal decoding apparatus 500. ing. Here, the first remix signal decoding apparatus 500 includes an additional information decoding unit 503 and a remixing rendering unit 506. Similarly, the first remix signal decoding apparatus 500 and the first remix signal decoding apparatus described with reference to FIG. Since the functions are substantially the same, detailed description thereof is omitted.

  The input unit 6110 is an input device for inputting a reproduction command for the mix signal. Of course, the input unit 6110 may be implemented by a direction movement key, a touch wheel, or the like in addition to a pointing device such as a mouse, a trackball, or a touch pad.

  The storage unit 6120 is a storage device that stores an image for each source signal. For example, musical instrument-specific images (piano image, violin image, flute image, etc.), and player-specific images (John Myeong-hoon photo, Jo Sumi photo, etc.) can be stored.

  The control unit 6130 performs control so that the shape of the image of the source signal changes with time as the level of the source signal corresponding to the mix signal changes with time. Here, the image may be an image for each source signal inquired from the storage unit 6120 based on the source signal information si received from the additional information encoding unit 503 or the remixing rendering unit 505 of the first remix signal decoding apparatus 500. . The time level of the source signal may be extracted from the source signal level information sl received from the remixing / rendering unit 505 of the first remix signal decoding apparatus 500.

  For example, if the total playback time is 3 minutes 45 seconds, each source signal (piano, violin, flute) every second from playback start (t = 0 minutes 0 seconds) to playback completion (t = 3 minutes 45 seconds) Are detected and the image (piano image, violin image, flute image) corresponding to each level is changed in shape (size, hue, brightness, position in the screen, etc.) and displayed. If the level of the piano (or violin, etc.) at a specific time is large, the size of the piano image (or violin image, etc.) can be changed greatly. If the level of the piano (or violin, etc.) is high at a specific time, the piano The brightness of the image (or violin etc.) can be changed brightly.

  The display 6140 is a display device for displaying an image corresponding to the source signal.

  FIG. 41 is a diagram showing a configuration of an interface display device according to still another embodiment of the present invention. Referring to the figure, an interface display device 6200 according to another embodiment of the present invention includes an input unit 6210, a storage unit 6220, a control unit 6230, and a display 6240, and is associated with the second remix signal decoding device 700. ing. Here, the first remix signal decoding apparatus 700 includes an additional information decoding unit 703, an additional information integration unit 705, and a remixing rendering unit 708. Similarly, the second remix signal described with reference to FIG. 7 above. Since the configuration and function of the signal decoding apparatus are substantially the same, a detailed description thereof will be omitted.

  Meanwhile, the interface display device 6200 according to another embodiment of the present invention may have a second remix signal instead of the first remix signal decoding device 500 when compared with the interface display device 6100 according to another embodiment of the present invention. The configuration is substantially the same as that of the interface display device 6100 according to the embodiment of the present invention except that it is related to the decoding device 700 and is different in that the composite object signal 701 is input instead of the mix signal 501. A description of the elements is omitted.

  FIG. 42 is a flowchart illustrating an interface display method according to another embodiment of the present invention. The term mix signal used in the following should be understood as a concept that includes even the concept of a composite object signal. The interface display method according to another embodiment of the present invention may be performed by the interface display device 6100 according to still another embodiment of the present invention described above, and the interface display device according to still another embodiment of the present invention. 6200 may be performed.

  Referring to FIG. 42, when the user inputs a playback command for the specific mix signal using the input device, the interface display device receives the playback command (S610). Thereafter, the interface display device receives source signal information corresponding to the mix signal (S620). The interface display apparatus inquires an image corresponding to each source signal based on the source signal information received in operation S620 (operation S630).

  Thereafter, when the reproduction of the mix signal is started (“Yes” in step S640), the interface display device senses the level of the source signal corresponding to the mix signal (or the level for each subband) by time (step S650). Thereafter, the shape (size, hue, brightness, position in the screen, etc.) inquired in step S630 is changed according to time according to the level inquired in step S640 (level by subband) (S660). Stage). FIG. 43 shows an example of a screen in which the image size changes in proportion to the source signal level. Looking at the size of each object image in FIG. 43, when the current playback time is “t1”, the level of female vocalists is high, but the level of instruments other than female vocalists (piano, violin, flute) is It turns out that it is low. It can be seen that when the current playback time is 't2', the level of the female vocalist is lower than that of the other instruments, and when the current playback time is 't3', only the level of the piano and female vocalist is high. An example of a screen in which the brightness of the image changes in proportion to the level of the source signal is shown in FIG. Referring to FIG. 44, it can be seen that the brightness of the image for each object changes as the level of the source signal changes at times t1, t2, and t3, respectively.

  On the other hand, the size of the image corresponding to the source signal can be changed as the level of the source signal changes. For example, only the image of the source signal having the highest level at a specific time point is displayed, and the other source signals The image size can be set to 0 and not displayed on the screen. An example of a screen on which only an image having the highest level among the source signals is displayed is shown in FIGS. Referring to FIG. 45, at the playback time t1 from the start of playback, the piano level is the highest among the total of four source signals, so the image of the other source signals (violin, vocalist, flute) disappears and the piano It can be seen that only the image (photo of piano player Jung Mun-Hun) is displayed on the screen, and at the playback time t2, the level of the vocalist (girls) is the highest among the total of four source signals. It can be seen that only the image of a vocalist (girl) (photo of Cho Sumi, a vocalist) is displayed on the screen. Referring to FIG. 46, the image of the third singer with the highest level is placed in the center at the playback time t1 after the start of playback, and the image of the first singer with the highest level is centered at the playback time t2. It can be seen that they are arranged in the section.

  If the image is changed according to the subband level of the source signal, the shape of the image of the source signal can be partially changed. For example, if the horizontal axis of the image corresponds to the frequency band axis of the source signal and the vertical axis of the image corresponds to the level axis of the source signal, the level of the low frequency band is high and the level of the high frequency band is high. The left side of can be expressed lightly, and the right side of the image can be expressed darkly. The steps S650 and S660 are continued until the reproduction of the mix signal is completed (step S670).

  A user can receive remix information from a server that provides remix information (hereinafter referred to as a remix information providing server). The remix information providing server may provide a modified mix signal in which some of the source signals included in the mix signal are excluded. That is, when the mix signal is composed of a plurality of source signals, the remix information providing server provides a modified mix signal in which one or more source signals are excluded from the source signals constituting the mix signal. You can also Hereinafter, a mix signal from which some of the source signals included in the mix signal are excluded is referred to as a “modified mix signal” in order to distinguish it from the original mix signal. When the remix information providing server provides the modified mix signal, the user can first receive and listen to the modified mix signal before receiving the mix signal. If the user wants to hear the original mix signal that is not excluded from the source signal after listening to the modified mix signal, the original mix signal containing all the source signals is transmitted to the remix information providing server. Can be requested.

  The remix information providing server can provide remix information to the user free of charge, but can also provide part or all of the remix information to the user for a fee. When the remix information providing server provides the remix information to the user for a fee, the user can receive the remix information after paying the remix information usage fee. The remix information providing server can provide the user with the modified mix signal for free and the original mix signal for a fee. Hereinafter, FIGS. 47 to 52 will describe a method in which a user receives a modified mix signal in which some of the source signals are missing from the remix information providing server, and uses the modified mix signal. A payment method when the information providing server provides remix information for a fee will be described.

  FIG. 47 is a diagram illustrating a remix information providing system according to an embodiment of the present invention. 47, the remix information providing system includes a user terminal 7001, a remix information providing server 7002, a financial institution 7003, and a mobile communication carrier 7004. The user needs a mix signal or a source signal and additional information for this in order to remix the source signal. The additional information includes an encoder mix parameter generated by the encoding device, a blind mix parameter generated by the decoding device, a user mix parameter generated using control information input by the user, and the like. The user may adjust an icon representing each source signal via a user interface (not shown) included in the user terminal 7001 and adjust the source signal included in the remixed mix signal. it can. The user can change the size and position of the source signal, the voice characteristics, and the like by adjusting an icon representing the source signal. The icons may include emoticons that express human emotions, photographs and caricatures of singers and musical instrument players, or photographs and pictures of musical instruments. Below, the mix signal, source signal of each sound source, mix parameters for controlling the mix signal for each source signal, upmix parameters for upmixing the mix signal, icons for expressing the source signal, etc. All information necessary for remixing each signal is called remix information.

  The user accesses the remix information providing server 7002 using the user terminal 7001. The user terminal 7001 can perform wireless communication with other user terminals such as user PCs such as notebooks and desktops, mobile terminals such as mobile phones, PDA (personal digital assistants), PMP (portable multimedia player), and so on. Including a device capable of playing back. The user terminal 7001 selects desired remix information from the remix information providing server 7002. The remix information providing server 7002 retrieves remix information selected by the user from a database (not shown) and extracts it. The remix information providing server 7002 transmits the extracted remix information to the user terminal 7001. The remix information providing server 7002 may provide a modified mix signal in which a certain source signal is excluded from a specific section of the mix signal or a source signal included in the mix signal. For example, if a user wants to listen to a piano trio of 'Great Artist's Memories' of piano, violin, cello Tchaikovsky, the user first listens to the modified mix signal for it before listening to the mix signal be able to. When the user requests the remix information providing server 7002 for a modified mix signal for a piano trio of “Great Artist's Memories”, the remix information providing server 7002 may receive the “great artist” with the cello signal excluded. A modified mix signal called “memory” can be extracted from the database and transmitted to the user. The remix information providing server 7002 determines that the source signal included in the mix signal, i.e., piano, violin, cello signal, when the modified mix signal for the “great artist's memories” is not stored in the database. By excluding certain source signals, the mix signal can be variously modified. The remix information providing server 7002 can generate a modified mix signal by excluding a specific source signal from the source signal included in the mix signal randomly or in response to a user request. When the user requests to exclude the piano signal from the source signal included in the mix signal and generate a modified mix signal, the remix information providing server 7002 includes the modified violin and cello signal. A mix signal can be generated and transmitted to the user. The user receives the modified mix signal from the remix information providing server 7002 and listens to it. If the user wants to listen to the mixed signal after listening to the deformed mixed signal, the user requests the remix information providing server 7002 to transmit the mixed signal for the memory of the great artist. The user can listen to the modified mix signal or the mix signal received from the remix information providing server 7002 using the user terminal 7001. The user can use the mix signal or the like as the user terminal 7001, for example, as a call sound of the mobile terminal or a bell sound of the mobile terminal. Further, the user can also use the mix signal or the like as background music such as a home page. In addition, the user receives additional information on the mix signal or the modified mix signal from the remix information providing server 7002, and adjusts the mix signal or the like for each source signal by using the additional information, thereby generating a remixed mix signal. be able to.

  In some cases, the remix information providing server 7002 can provide all or part of the remix information for a fee. Basic remix information can be provided free of charge, and other remix information can be provided for a fee. If the remix information is charged for use, the remix information providing server 7002 requests the user to make a payment. In the settlement process, the remix information providing server 7002 can request authentication information from the user terminal 7001 in order to confirm whether the user is a valid user. In response to a request from the remix information providing server 7002, the user inputs user information for authentication to the user terminal 7001. The remix information providing server 7002 can receive user information input by the user from the user terminal 7001 and authenticate whether the user is a valid user.

  When the user intends to settle the remix information usage fee with a credit card, the remix information providing server 7002 transmits the user information to a financial institution 7003 such as a card company or a bank and requests confirmation of the validity of the user. The financial institution 7003 confirms whether or not the user is a valid user who has joined the financial institution using the user information, and transmits the result to the remix information providing server 7002. When the user is a valid user, the remix information providing server 7002 requests the financial institution 7003 to perform remix information usage fee settlement. The financial institution 7003 extracts the remix information usage fee from the user's account, and notifies the remix information providing server 7002 that the settlement has been completed. The financial institution 7003 transfers the extracted remix information usage fee to the account of the remix information providing server 7002 in real time or at regular time intervals. When the user intends to settle the remix information usage fee by the account transfer method, the financial institution 7003 notifies that the user has paid the remix information usage fee to the account of the remix information providing server 7002. The remix information providing server 7002 receives the notification of settlement completion from the financial institution 7003, and then extracts the remix information selected by the user and transmits it to the user terminal 7001. The user terminal 7001 may receive the remix information from the remix information providing server 7002 and listen to it as it is, but the user may modify the remix information and use it. In this case, the user terminal 7001 may include a signal decoding device for processing remixing. The user can receive the source signal, the mix signal, and the additional information together or separately, and use the user terminal 7001 to remix the source signal. When the user terminal 7001 receives only the mix signal from the remix information providing server 7002, the user terminal 7001 directly generates additional information for the mix signal, and adjusts the mix signal for each source signal using the generated additional information. it can. The user may adjust the mix signal according to the source signal to generate a remixed mix signal and upload the remixed mix signal to the remix information providing server 7002. In addition, the user can generate an icon representing the source signal or transform the icon received from the remix information providing server 7002 and upload the icon to the remix information providing server 7002. At this time, the remix information includes a remixed mix signal uploaded by a user, an icon that is transformed or generated, and uploaded. The remix information providing server 7002 stores the remixed mix signal uploaded by the user using the user terminal 7001 in a database (not shown). When another second user requests transmission of the remixed mix signal, the remix information providing server 7002 can transmit it to the second user terminal for a fee or free of charge. The remix information providing server 7002, the user who has uploaded the remixed mix signal or icon, etc. to the remix information providing server 7002, and the second user who intends to use the remixed mix signal or icon, etc., interact with each other according to a predetermined contract. You can pay the remix information usage fee. When the remix information providing server 7002 transmits the remixed mix signal to the second user for a fee, the remix information providing server 7002 can request the second user to make a settlement using the method described above. The remix information providing server 7002 may pay a fixed amount of the remix information usage fee paid by the second user to the user who uploaded the remixed mix signal or icon to the remix information providing server 7002.

  The user can also settle the remix information usage fee using a user terminal 7001 such as a mobile phone. When the user inputs a user terminal 7001 number such as a mobile phone to the remix information providing server 7002, the remix information providing server 7002 transmits the user terminal 7001 number of the user to the mobile communication carrier 7004, and authentication for user authentication. Request a number. The mobile communication carrier 7004 transmits the authentication number to the remix information providing server 7002 and the user terminal 7001 in response to a request from the remix information providing server 7002. The user inputs the authentication number transmitted to the user terminal 7001 to the remix information providing server 7002 using the user terminal 7001. When the authentication number input by the user matches the authentication number transmitted from the mobile communication carrier 7004, the remix information providing server 7002 determines that the user is a valid user. When the remix information providing server 7002 determines that the user is an effective user, the remix information providing server 7002 requests the mobile communication carrier 7004 to perform remix information usage fee settlement. The mobile communication carrier 7004 adds the remix information usage fee to the usage fee of the user terminal 7001. When the payment for the remix information usage fee is completed, the mobile communication carrier 7004 notifies the remix information providing server 7002 that the payment has been completed. The remix information providing server 7002 transmits the remix information selected by the user to the user terminal 7001 after receiving the notification of completion of settlement. The user generates a remixed mix signal using the remix information transmitted to the user terminal 7001. The user can generate a unique mix signal by adjusting the size, position, tone, and the like of the source signal using the user terminal 7001. The user can upload the remixed mix signal to the remix information providing server 7002 using the user terminal 7001. As described above, the second user can download and use the remixed mix signal uploaded to the remix information providing server 7002 by another user.

  FIG. 48 is a block diagram of a remix information providing server according to an embodiment of the present invention. Referring to FIG. 48, the remix information providing server 7002 includes a storage device 7001, a central processing unit (CPU) 7104, an input / output device 7108, and a communication device 7109. The central processing unit 7104 includes an ALU (Arithmetic Logic Unit) 7105 for performing calculations, a register 7106 for temporarily storing data and instruction words, and a controller 7107 for controlling the operation of the remix information providing server 7002. The memory system 7001 usually includes a high-speed main storage device 7102 which is a storage medium form such as RAM (Random Access Memory) and ROM (Read Only Memory), a floppy (registered trademark) disk, a hard disk, a tape, a CD-ROM, It includes an auxiliary storage device 7103 in the form of a long-term storage medium such as a flash memory and a device for storing data using electrical, magnetic, optical, or other storage media. The main memory 7102 may include a video display memory for displaying remix information such as a list of mixed signals or transformed mix signals and icons through a display device. The input / output device 7108 may include a keyboard, a mouse, a physical transducer such as a touch screen or a microphone, and the like. As the communication device 7109, a device such as a network interface or an RF transmission / reception antenna for performing wired or wireless communication with the user terminal 7001 can be used.

  FIG. 49 is a module configuration diagram of a remix information providing server according to an embodiment of the present invention. Referring to FIG. 49, the remix information providing server 7002 includes a transmission / reception processing module 7213 for processing communication with the user terminal 7001, the financial institution 7003, and the mobile communication carrier 7004, a mix signal, a modified mix signal, and the like. A database management module 7214 that stores and manages remix information, a mix signal deformation module 7215 that generates a deformed mix signal, and a billing process that authenticates whether the user is valid and processes payment when the remix information is charged A module 7216 can be included. The overall configuration of the remix information providing server through such application modules will be described as follows. The remix information providing server can use various OS (Operating System) as the OS of the system. Such an OS provides a high-level instruction word to an API (Application Program Interface) 7201 to control the operation of each application module 7204. The remix information providing server identifies each application module 7204 corresponding to the high-level instruction word provided from the API 7201, decodes the high-level instruction word, and provides it to the corresponding place. 7202 is included. The application module control unit 7203 controls the operation of the application module 7204 according to the instruction word provided from the high level instruction word processing unit 7202. That is, the high-level instruction word processing unit 7202 identifies whether an application module 7204 corresponding to the high-level instruction word provided from the API 7201 exists. If the corresponding application module 7204 exists, the application module 7204 It is decoded into a recognizable command word and transmitted to the corresponding mapping unit, or message transmission is controlled. Therefore, the application module control unit 7203 includes mapping units 7205, 7207, 7209, 7211 and interface units 7206, 7208, 7210, 7212 for the transmission / reception processing module 7213, the database management module 7214, the mix signal transformation module 7215, and the billing processing module 7216. Includes each.

  The transmission / reception processing module mapping unit 7205 receives a high-level instruction word for communication with the user terminal 7001, the financial institution 7003 or the mobile communication carrier 7004 from the high-level instruction word processing unit 7202, and the transmission / reception processing module 7213 It is mapped to a device level that can be processed and provided to the transmission / reception processing module 7213 through the transmission / reception processing module interface unit 7206. The database management module mapping unit 7207 and the database management module interface unit 7208 store the mix signal or the modified mix signal, and extract the mix signal or the modified mix signal when requested from the user terminal 7001. The database management module mapping unit 7207 receives a high-level instruction word for using the database management module 7214 from the high-level instruction word processing unit 7202, maps it to a device-level instruction word, and transmits the database through the database management module interface unit 7208. To the management module 7214. The mix signal modification module 7215 excludes a certain source signal included in the mix signal when the mix signal needs to be modified, for example, when a modified mix signal is requested by the user. Generate a signal. The mixed signal modification module mapping unit 7209 receives the high-level instruction word applied from the high-level instruction word processing unit 7202 and maps it to a device-level instruction word that can be recognized by the modified signal modification module 7215. The device level instruction word is provided to the mix signal modification module 7215 through the mix signal modification module interface unit 7210. The billing processing module 7216 functions to check whether the user is a valid user and to bill the remix information usage fee if the user is a valid user. In addition, the billing processing module 7216 may use the remix information providing server 7002, the user who uploaded the remixed mix signal, if the second user wishes to use the remixed mix signal generated and uploaded by another user, and It fulfills the function of processing the remix information usage fee between the second users. The charging processing module mapping unit 7211 receives the high level command word applied from the high level command word processing unit 7202 and maps it to a device level command word that can be recognized by the charging processing module 7216. The device level instruction word is provided to the charging processing module 7216 through the charging processing module interface unit 7212.

  FIG. 50 is a flowchart illustrating a method for providing remix information by a remix information providing server according to an embodiment of the present invention. Referring to FIG. 50, the user connects to the remix information providing server 7002 using the user terminal 7001. The user terminal 7001 selects remix information from the remix information providing server 7002 (step S701). The remix information providing server 7002 may transmit the user-selected remix information to the user terminal 7001 free of charge or may transmit it for a fee. The remix information providing server 7002 determines whether the remix information selected by the user is charged or free (step S703). If the remix information selected by the user is charged, the remix information providing server 7002 requests payment from the user (step S705). For settlement, the remix information providing server 7002 can perform user authentication to confirm whether the user is a valid user. The remix information providing server 7002 can request the user terminal 7001 to input user authentication information for user authentication such as a credit card number, an account number, a resident number, and a password. When the user inputs user authentication information using the user terminal 7001, the remix information providing server 7002 transmits the user authentication information input by the user to the financial institution 7003, the mobile communication carrier 7004, or the like, and whether or not the user is valid. Request to confirm. The financial institution 7003 or the mobile communication carrier 7004 searches the user authentication information transmitted by the remix information providing server 7002 from a database (not shown), and notifies the remix information providing server 7002 of the result. When the remix information providing server 7002 determines that the user is valid, the remix information providing server 7002 transmits settlement information to the financial institution 7003 or the mobile communication carrier 7004 and requests settlement. The remix information providing server 7002 can include a mix signal transformation unit (not shown). When the settlement is completed (“Yes” in step 707), the remix information providing server 7002 determines whether or not the remix information selected by the user is a modified mix signal from which some source signals are excluded (step 707). S709). When the remix information desired by the user is a deformed mix signal, the remix information providing server 7002 deforms the mix signal using a mix signal deforming unit or extracts a deformed mix signal that has already been stored. (Step S711). The remix information providing server 7002 provides the modified mix signal to the user terminal 7001 (step S715). If the remix information selected by the user is not a modified mix signal, the remix information providing server 7002 extracts remix information already stored (step S713). The remix information providing server 7002 provides the extracted remix information to the user terminal 7001 (step S715).

  FIG. 51 is a flowchart illustrating a method in which a user terminal receives remix information according to an embodiment of the present invention. Referring to FIG. 51, the user connects to the remix information providing server 7002 using the user terminal 7001 (step S801). The user uses the user terminal 7001 to select desired information from the remix information provided by the remix information providing server 7002 (step S803). As described above, the remix information includes a mix signal, a modified mix signal, a source signal, additional information, an icon representing the source signal, an emoticon, and the like. The remix information includes remix information uploaded by other users to the remix information providing server 7002. The user terminal 7001 determines whether the remix information selected by the user is charged (step S805). If the selected remix information is charged, the user pays a remix information usage fee (step S807). The remix information usage fee can vary depending on the type, capacity, bit rate, etc. of the remix information. For example, when the user wants to receive only a specific source signal, the user can pay a lower usage fee than when receiving a mix signal including the source signal. If the user receives additional information about the mix signal or the source signal not included in the mix signal, the amount that the user settles can vary. When the settlement is completed, the user receives remix information from the remix information providing server 7002 (step S809).

  FIG. 52 is a flowchart illustrating a method for transmitting remix information to a remix information providing server by a user according to an embodiment of the present invention. Referring to FIG. 52, the user receives remix information using user terminal 7001. The remix information can be received from an encoding device such as the remix information providing server 7002. The user terminal 7001 can include a remix rendering unit (not shown) that remixes a source signal included in the mix signal using the remix information and control information. When the user terminal 7001 includes a remix rendering unit, the user can directly generate remix information using the mix signal, the source signal, and additional information related thereto using the user terminal 7001. The user can also generate new remix information by transforming the remix information stored in the user terminal 7001 using the remix rendering unit (step S901). That is, the user can generate a remixed mix signal by adjusting the size, position, tone color, and the like of the source signal using the remix information that the user has. In addition, the user can change an icon or emoticon representing the source signal, or directly generate an icon or the like. The user transmits the generated remix information to the remix information providing server 7002 and uploads it (step S903). The remix information providing server 7002 stores the new remix information uploaded by the user in the database, and when the second user desires it, extracts it from the database and transmits it to the second user terminal.

The present invention can be applied to an input device or a remote controller of a media signal player. Further, the present invention can be applied to an interface control device of a media signal player. The present invention can be applied to a media signal information providing server or the like.
Although the present invention has been described above based on the embodiments and the drawings limited to the above, the present invention is not limited to them. For those having ordinary knowledge in the technical field to which the present invention belongs, the present invention is not limited thereto. It is apparent that various modifications and changes can be made within the scope equivalent to the technical idea of the present invention and the scope of the appended claims.

Claims (8)

An input section;
A pointing device or a direction key,
The input signal received through the input unit is converted into a menu display command for each source, source information corresponding to the mix signal is received, and output according to the source information according to the menu display command for each source is received. A control unit configured to control the menu display screen;
A display configured to output the menu display screen according to source;
Including
The source information is obtained by decoding an additional information signal, the mix signal is a mono or stereo channel signal including at least one source signal, and the source information is the at least one in the mix signal. Related to one source signal ,
The control unit is configured to convert the input signal received through the pointing device or the direction movement key into a source level adjustment command,
The additional information signal is used to decode the mix signal and has a gain factor effective to control panning and gain of each source signal of the decoded mix signal. Input device.   The input apparatus according to claim 1, further comprising an additional information decoding unit configured to obtain the source information from the additional information signal. A remix rendering unit configured to render the at least one source signal of the mix signal based on the source information and control information from the control unit;
The input device according to claim 2 , wherein the control unit generates the control information based on an input signal via the source-specific menu display screen. The input device according to claim 2 , wherein the source information includes at least one metadata associated with the source signal and gain information of the at least one source signal. An input configured to select one of a plurality of mix signals that are mono or stereo channel signals including at least one source signal;
A pointing device or a direction key,
A controller configured to obtain source information corresponding to a mix signal and display an icon corresponding to the at least one source signal based on the source information;
A display unit configured to display the icon corresponding to the source signal;
Including
The source information is obtained by decoding an additional information signal, the source information being related to the at least one source signal in the mix signal ;
The control unit is configured to convert an input signal received through the pointing device or a direction movement key into a source level adjustment command,
The additional information signal is used to decode the mix signal and has a gain factor effective to control panning and gain of each source signal of the decoded mix signal. Mix signal interface display device. The mixed signal interface display apparatus according to claim 5 , further comprising an additional information decoding unit configured to obtain the source information from the additional information signal. A remix rendering unit configured to render the at least one source signal of the mix signal based on the source information and control information from the control unit;
The mixed signal interface display device according to claim 6 , wherein the control unit generates the control information based on an input signal via a menu display screen for each source. 8. The mixed signal interface display apparatus according to claim 6 , wherein the source information includes at least one metadata associated with the source signal and gain information of the at least one source signal.

Images