US20090028344A1 - Method and Apparatus for Processing a Media Signal - Google Patents
Method and Apparatus for Processing a Media Signal Download PDFInfo
- Publication number
- US20090028344A1 US20090028344A1 US12/161,560 US16156007A US2009028344A1 US 20090028344 A1 US20090028344 A1 US 20090028344A1 US 16156007 A US16156007 A US 16156007A US 2009028344 A1 US2009028344 A1 US 2009028344A1
- Authority
- US
- United States
- Prior art keywords
- information
- rendering
- filter
- signal
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 203
- 238000012545 processing Methods 0.000 title claims abstract description 33
- 238000009877 rendering Methods 0.000 claims abstract description 360
- 238000009499 grossing Methods 0.000 claims abstract description 83
- 238000013507 mapping Methods 0.000 claims abstract description 79
- 230000000694 effects Effects 0.000 claims abstract description 24
- 238000001914 filtration Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 64
- 238000010586 diagram Methods 0.000 description 44
- 238000005192 partition Methods 0.000 description 24
- 230000005236 sound signal Effects 0.000 description 23
- 230000004044 response Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 238000006243 chemical reaction Methods 0.000 description 8
- 238000001228 spectrum Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 102100040836 Claudin-1 Human genes 0.000 description 1
- 101100113671 Homo sapiens CLDN1 gene Proteins 0.000 description 1
- 101100113675 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) CLD1 gene Proteins 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S3/00—Systems employing more than two channels, e.g. quadraphonic
- H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/007—Two-channel systems in which the audio signals are in digital form
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S3/00—Systems employing more than two channels, e.g. quadraphonic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- the present invention relates to an apparatus for processing a media signal and method thereof, and more particularly to an apparatus for generating a surround signal by using spatial information of the media signal and method thereof.
- various kinds of apparatuses and methods have been widely used to generate a multi-channel media signal by using spatial information for the multi-channel media signal and a downmix signal, in which the downmix signal is generated by downmixing the multi-channel media signal into mono or stereo signal.
- the above methods and apparatuses are not usable in environments unsuitable for generating a multi-channel signal. For instance, they are not usable for a device capable of generating only a stereo signal. In other words, there exists no method or apparatus for generating a surround signal, in which the surround signal has multi-channel features in the environment incapable of generating a multi-channel signal by using spatial information of the multi-channel signal.
- the present invention is directed to an apparatus for processing a media signal and method thereof that substantially obviate one or more of the problems due to limitations and disadvantages of the related art.
- An object of the present invention is to provide an apparatus for processing a media signal and method thereof, by which the media signal can be converted to a surround signal by using spatial information for the media signal.
- a method of processing a signal includes of: generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources; generating sub-rendering information by applying filter information giving a surround effect to the source mapping information per the source; generating rendering information for generating a surround signal by integrating at least one of the sub-rendering information; and generating the surround signal by applying the rendering information to a downmix signal generated by downmixing the multi-sources.
- an apparatus for processing a signal includes a source map ping unit generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources; a sub-rendering information generating unit generating sub-rendering information by applying filter information having a surround effect to the source mapping information per the source; an integrating unit generating rendering information for generating a surround signal by integrating the at least one of the sub-rendering information; and a rendering unit generating the surround signal by applying the rendering information to a downmix signal generated by downmixing the multi-sources.
- a signal processing apparatus and method according to the present invention enable a decoder, which receives a bitstream including a downmix signal generated by downmixing a multi-channel signal and spatial information of the multi-channel signal, to generate a signal having a surround effect in environments in incapable of recovering the multi-channel signal.
- FIG. 1 is a block diagram of an audio signal encoding apparatus and an audio signal decoding apparatus according to one embodiment of the present invention
- FIG. 2 is a structural diagram of a bitstream of an audio signal according to one embodiment of the present invention.
- FIG. 3 is a detailed block diagram of a spatial information converting unit according to one embodiment of the present invention.
- FIG. 4 and FIG. 5 are block diagrams of channel configurations used for source mapping process according to one embodiment of the present invention.
- FIG. 6 and FIG. 7 are detailed block diagrams of a rendering unit for a stereo downmix signal according to one embodiment of the present invention.
- FIG. 8 and FIG. 9 are detailed block diagrams of a rendering unit for a mono downmix signal according to one embodiment of the present invention.
- FIG. 10 and FIG. 11 are block diagrams of a smoothing unit and an expanding unit according to one embodiment of the present invention.
- FIG. 12 is a graph to explain a first smoothing method according to one embodiment of the present invention.
- FIG. 13 is a graph to explain a second smoothing method according to one embodiment of the present invention.
- FIG. 14 is a graph to explain a third smoothing method according to one embodiment of the present invention.
- FIG. 15 is a graph to explain a fourth smoothing method according to one embodiment of the present invention.
- FIG. 16 is a graph to explain a fifth smoothing method according to one embodiment of the present invention.
- FIG. 17 is a diagram to explain prototype filter information corresponding to each channel
- FIG. 18 is a block diagram for a first method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention.
- FIG. 19 is a block diagram for a second method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention.
- FIG. 20 is a block diagram for a third method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention.
- FIG. 21 is a diagram to explain a method of generating a surround signal in a rendering unit according to one embodiment of the present invention.
- FIG. 22 is a diagram for a first interpolating method according to one embodiment of the present invention.
- FIG. 23 is a diagram for a second interpolating method according to one embodiment of the present invention.
- FIG. 24 is a diagram for a block switching method according to one embodiment of the present invention.
- FIG. 25 is a block diagram for a position to which a window length decided by a window length deciding unit is applied according to one embodiment of the present invention.
- FIG. 26 is a diagram for filters having various lengths used in processing an audio signal according to one embodiment of the present invention.
- FIG. 27 is a diagram for a method of processing an audio signal dividedly by using a plurality of subfilters according to one embodiment of the present invention.
- FIG. 28 is a block diagram for a method of rendering partition rendering information generated by a plurality of subfilters to a mono downmix signal according to one embodiment of the present invention
- FIG. 29 is a block diagram for a method of rendering partition rendering information generated by a plurality of subfilters to a stereo downmix signal according to one embodiment of the present invention.
- FIG. 30 is a block diagram for a first domain converting method of a downmix signal according to one embodiment of the present invention.
- FIG. 31 is a block diagram for a second domain converting method of a downmix signal according to one embodiment of the present invention.
- FIG. 1 is a block diagram of an audio signal encoding apparatus and an audio signal decoding apparatus according to one embodiment of the present invention.
- an encoding apparatus 10 includes a downmixing unit 100 , a spatial information generating unit 200 , a downmix signal encoding unit 300 , a spatial information encoding unit 400 , and a multiplexing unit 500 .
- the downmixing unit 100 downmixes the inputted signal into a downmix signal.
- the downmix signal includes mono, stereo and multi-source audio signal.
- the source includes a channel and, in convenience, is represented as a channel in the following description.
- the mono or stereo downmix signal is referred to as a reference. Yet, the present invention is not limited to the mono or stereo downmix signal.
- the encoding apparatus 10 is able to optionally use an arbitrary downmix signal directly provided from an external environment.
- the spatial information generating unit 200 generates spatial information from a multi-channel audio signal.
- the spatial information can be generated in the course of a downmixing process.
- the generated downmix signal and spatial information are encoded by the downmix signal encoding unit 300 and the spatial information encoding unit 400 , respectively and are then transferred to the multiplexing unit 500 .
- spatial information means information necessary to generate a multi-channel signal from upmixing a downmix signal by a decoding apparatus, in which the downmix signal is generated by downmixing the multi-channel signal by an encoding apparatus and transferred to the decoding apparatus.
- the spatial information includes spatial parameters.
- the spatial parameters include CLD (channel level difference) indicating an energy difference between channels, ICC (inter-channel coherences) indicating a correlation between channels, CPC (channel prediction coefficients) used in generating three channels from two channels, etc.
- ‘downmix signal encoding unit’ or ‘downmix signal decoding unit’ means a codec that encodes or decodes an audio signal instead of spatial information.
- a downmix audio signal is taken as an example of the audio signal instead of the spatial information.
- the downmix signal encoding or decoding unit may include MP3, AC-3, DTS, or AAC.
- the downmix signal encoding or decoding unit may include a codec of the future as well as the previously developed codec.
- the multiplexing unit 500 generates a bitstream by multiplexing the downmix signal and the spatial information and then transfers the generated bitstream to the decoding apparatus 20 . Besides, the structure of the bitstream will be explained in FIG. 2 later.
- a decoding apparatus 20 includes a demultiplexing unit 600 , a downmix signal decoding unit 700 , a spatial information decoding unit 800 , a rendering unit 900 , and a spatial information converting unit 1000 .
- the demultiplexing unit 600 receives a bitstream and then separates an encoded downmix signal and an encoded spatial information from the bitstream. Subsequently, the downmix signal decoding unit 700 decodes the encoded downmix signal and the spatial information decoding unit 800 decodes the encoded spatial information.
- the spatial information converting unit 1000 generates rendering information applicable to a downmix signal using the decoded spatial information and filter information.
- the rendering information is applied to the downmix signal to generate a surround signal.
- the surround signal is generated in the following manner.
- a process for generating a downmix signal from a multi-channel audio signal by the encoding apparatus 10 can include several steps using an OTT (one-to-two) or TTT (three-to-three) box.
- spatial information can be generated from each of the steps.
- the spatial information is transferred to the decoding apparatus 20 .
- the decoding apparatus 20 then generates a surround signal by converting the spatial information and then rendering the converted spatial information with a downmix signal.
- the present invention relates to a rendering method including the steps of extracting spatial information for each upmixing step and performing a rendering by using the extracted spatial information.
- HRTF head-related transfer functions
- the spatial information is a value applicable to a hybrid domain as well. So, the rendering can be classified into the following types according to a domain.
- the first type is that the rendering is executed on a hybrid domain by having a downmix signal pass through a hybrid filterbank. In this case, a conversion of domain for spatial information is unnecessary.
- the second type is that the rendering is executed on a time domain.
- the second type uses a fact that a HRTF filter is modeled as a FIR (finite inverse response) filter or an IIR (infinite inverse response) filter on a time domain. So, a process for converting spatial information to a filter coefficient of time domain is needed.
- the third type is that the rendering is executed on a different frequency domain.
- the rendering is executed on a DFT (discrete Fourier transform) domain.
- DFT discrete Fourier transform
- a process for transforming spatial information into a corresponding domain is necessary.
- the third type enables a fast operation by replacing a filtering on a time domain into an operation on a frequency domain.
- filter information is the information for a filter necessary for processing an audio signal and includes a filter coefficient provided to a specific filter. Examples of the filter information are explained as follows. First of all, prototype filter information is original filter information of a specific filter and can be represented as GL_L or the like. Converted filter information indicates a filter coefficient after the prototype filter information has been converted and can be represented as GL_L or the like. Sub-rendering information means the filter information resulting from spatializing the prototype filter information to generate a surround signal and can be represented as FL_L 1 or the like. Rendering information means the filter information necessary for executing rendering and can be represented as HL_L or the like.
- Interpolated/smoothed rendering information means the filter information resulting from interpolation/smoothing the rendering information and can be represented as HL_L or the like.
- the above filter informations are referred to.
- the present invention is not restricted by the names of the filter informations.
- HRTF is taken as an example of the filter information.
- the present invention is not limited to the HRTF.
- the rendering unit 900 receives the decoded downmix signal and the rendering information and then generates a surround signal using the decoded downmix signal and the rendering information.
- the surround signal may be the signal for providing a surround effect to an audio system capable of generating only a stereo signal.
- the present invention can be applied to various systems as well as the audio system capable of generating only the stereo signal.
- FIG. 2 is a structural diagram for a bitstream of an audio signal according to one embodiment of the present invention, in which the bitstream includes an encoded downmix signal and encoded spatial information.
- a 1-frame audio payload includes a downmix signal field and an ancillary data field.
- Encoded spatial information can be stored in the ancillary data field. For instance, if an audio payload is 48 ⁇ 128 kbps, spatial information can have a range of 5 ⁇ 32 kbps. Yet, no limitations are put on the ranges of the audio payload and spatial information.
- FIG. 3 is a detailed block diagram of a spatial information converting unit according to one embodiment of the present invention.
- a spatial information converting unit 1000 includes a source mapping unit 1010 , a sub-rendering information generating unit 1020 , an integrating unit 1030 , a processing unit 1040 , and a domain converting unit 1050 .
- the source mapping unit 101 generates source mapping information corresponding to each source of an audio signal by executing source mapping using spatial information.
- the source mapping information means per-source information generated to correspond to each source of an audio signal by using spatial information and the like.
- the source includes a channel and, in this case, the source mapping information corresponding to each channel is generated.
- the source mapping information can be represented as a coefficient. And, the source mapping process will be explained in detail later with reference to FIG. 4 and FIG. 5 .
- the sub-rendering information generating unit 1020 generates sub-rendering information corresponding to each source by using the source mapping information and the filter information. For instance, if the rendering unit 900 is the HRTF filter, the sub-rendering information generating unit 1020 is able to generate sub-rendering information by using HRTF filter information.
- the integrating unit 1030 generates rendering information by integrating the sub-rendering information to correspond to each source of a downmix signal.
- the rendering information which is generated by using the spatial information and the filter information, means the information to generate a surround signal by being applied to the downmix signal.
- the rendering information includes a filter coefficient type. The integration can be omitted to reduce an operation quantity of the rendering process. Subsequently, the rendering information is transferred to the processing unit 1042 .
- the processing unit 1042 includes an interpolating unit 1041 and/or a smoothing unit 1042 .
- the rendering information is interpolated by the interpolating unit 1041 and/or smoothed by the smoothing unit 1042 .
- the domain converting unit 1050 converts a domain of the rendering information to a domain of the downmix signal used by the rendering unit 900 . And, the domain converting unit 1050 can be provided to one of various positions including the position shown in FIG. 3 . So, if the rendering information is generated on the same domain of the rendering unit 900 , it is able to omit the domain converting unit 1050 . The domain-converted rendering information is then transferred to the rendering unit 900 .
- the spatial information converting unit 1000 can include a filter information converting unit 1060 .
- the filter information converting unit 1060 is provided within the spatial information converting unit 100 .
- the filter information converting unit 1060 can be provided outside the spatial information converting unit 100 .
- the filter information converting unit 1060 is converted to be suitable for generating sub-rendering information or rendering information from random filter information, e.g., HRTF.
- the converting process of the filter information can include the following steps.
- a step of matching a domain to be applicable is included. If a domain of filter information does not match a domain for executing rendering, the domain matching step is required. For instance, a step of converting time domain HRTF to DFT, QMF or hybrid domain for generating rendering information is necessary.
- a coefficient reducing step can be included.
- a method of reducing a filter coefficient to be stored while maintaining filter characteristics in the domain converting process can be used. For instance, the HRTF response can be converted to a few parameter value. In this case, a parameter generating process and a parameter value can differ according to an applied domain.
- the downmix signal passes through a domain converting unit 1110 and/or a decorrelating unit 1200 before being rendered with the rendering information.
- the domain converting unit 1110 converts the domain of the downmix signal in order to match the two domains together.
- the decorrelating unit 1200 is applied to the domain-converted downmix signal. This may have an operational quantity relatively higher than that of a method of applying a decorrelator to the rendering information. Yet, it is able to prevent distortions from occurring in the process of generating rendering information.
- the decorrelating unit 1200 can include a plurality of decorrelators differing from each other in characteristics if an operational quantity is allowable. If the downmix signal is a stereo signal, the decorrelating unit 1200 may not be used. In FIG. 3 , in case that a domain-converted mono downmix signal, i.e., a mono downmix signal on a frequency, hybrid, QMF or DFT domain is used in the rendering process, a decorrelator is used on the corresponding domain.
- the present invention includes a decorrelator used on a time domain as well.
- a mono downmix signal before the domain converting unit 1100 is directly inputted to the decorrelating unit 1200 .
- a first order or higher IIR filter (or FIR filter) is usable as the decorrelator.
- the rendering unit 900 generates a surround signal using the downmix signal, the decorrelated downmix signal, and the rendering information. If the downmix signal is a stereo signal, the decorrelated downmix signal may not be used. Details of the rendering process will be described later with reference to FIGS. 6 to 9 .
- the surround signal is converted to a time domain by an inverse domain converting unit 1300 and then outputted. If so, a user is able to listen to a sound having a multi-channel effect though stereophonic earphones or the like.
- FIG. 4 and FIG. 5 are block diagrams of channel configurations used for source mapping process according to one embodiment of the present invention.
- a source mapping process is a process for generating source mapping information corresponding to each source of an audio signal by using spatial information.
- the source includes a channel and source mapping information can be generated to correspond to the channels shown in FIG. 4 and FIG. 5 .
- the source mapping information is generated in a type suitable for a rendering process.
- a downmix signal is a mono signal, it is able to generate source mapping information using spatial information such as CLD 1 ⁇ CLD 5 , ICC 1 ⁇ ICC 5 , and the like.
- the process for generating the source mapping information is variable according to a tree structure corresponding to spatial information, a range of spatial information to be used, and the like.
- the downmix signal is a mono signal for example, which does not put limitation of the present invention.
- Lo L*GL — L′+C*GC — L′+R*GR — L′+Ls*GLs — L′+Rs*GRs — L′
- the operator ‘*’ indicates a product on a DFT domain and can be replaced by a convolution on a QMF or time domain.
- the present invention includes a method of generating the L, C, R, Ls and Rs by source mapping information using spatial information or by source mapping information using spatial information and filter information.
- source mapping information can be generated using CLD of spatial information only or CLD and ICC of spatial information. The method of generating source mapping information using the CLD only is explained as follows.
- source mapping information is generated using CLD only, a 3-dimensional effect may be reduced. So, it is able to generate source mapping information using ICC and/or decorrelator. And, a multi-channel signal generated by using a decorrelator output signal dx(m) can be expresses as Math Figure 4.
- ‘A’, ‘B’ and ‘C’ are values that can be represented by using CLD and ICC.
- ‘d 0 ’ to ‘d 3 ’ indicate decorrelators.
- ‘m’ indicates a mono downmix signal. Yet, this method is unable to generate source mapping information such as D_L, D_R, and the like.
- the ‘dx’ is usable for a process for generating sub-rendering filter information according to Math Figure 5.
- HM — L FL — L — M+FR — L — M+FC — L — M+FLS — L — M+FRS — L — M+FLFE — L — M
- HM — R FL — R — M+FR — R — M+FC — R — M+FLS — R — M+FRS — R — M+FLFE — R — M
- HDx — L FL — L — Dx+FR — L — Dx+FC — L — Dx+FLS — L — Dx+FRS — L — Dx+FLFE — L — Dx
- the first method of generating the source mapping information using the CLD, ICC and/or decorrelators handles a dx output value, i.e., ‘dx(m)’ as an independent input, which may increase an operational quantity.
- a second method of generating source mapping information using CLD, ICC and/or decorrelators employs decorrelators applied on a frequency domain.
- the source mapping information can be expresses as Math Figure 7.
- a third method of generating source mapping information using CLD, ICC and/or decorrelators employs decorrelators having the all-pass characteristic as the decorrelators of the second method.
- the all-pass characteristic means that a size is fixed with a phase variation only.
- the present invention can use decorrelators having the all-pass characteristic as the decorrelators of the first method.
- a fourth method of generating source mapping information using CLD, ICC and/or decorrelators carries out decorrelation by using decorrelators for the respective channels (e.g., L, R, C, Ls, Rs, etc.) instead of using ‘d 0 ’ to ‘d 3 ’ of the second method.
- the source mapping information can be expressed as Math Figure 8.
- ‘k’ is an energy value of a decorrelated signal determined from CLD and ICC values.
- ‘d_L’, ‘d_R’, ‘d_C’, ‘d_Ls’ and ‘d_Rs’ indicate decorrelators applied to channels, respectively.
- a fifth method of generating source mapping information using CLD, ICC and/or decorrelators maximizes a decorrelation effect by configuring ‘d_L’ and ‘d_R’ symmetric to each other in the fourth method and configuring ‘d_Ls’ and ‘d_Rs’ symmetric to each other in the fourth method.
- a sixth method of generating source mapping information using CLD, ICC and/or decorrelators is to configure the ‘d_L’ and ‘d_Ls’ to have a correlation in the fifth method. And, the ‘d_L’ and ‘d_C’ can be configured to have a correlation as well.
- a seventh method of generating source mapping information using CLD, ICC and/or decorrelators is to use the decorrelators in the third method as a serial or nested structure of the all-pas filters.
- the seventh method utilizes a fact that the all-pass characteristic is maintained even if the all-pass filter is used as the serial or nested structure. In case of using the all-pass filter as the serial or nested structure, it is able to obtain more various kinds of phase responses. Hence, the decorrelation effect can be maximized.
- An eighth method of generating source mapping information using CLD, ICC and/or decorrelators is to use the related art decorrelator and the frequency-domain decorrelator of the second method together.
- a multi-channel signal can be expressed as Math Figure 9.
- [ L R C LFE Ls Rs ] [ A L ⁇ ⁇ 1 + K L ⁇ d L A R ⁇ ⁇ 1 + K R ⁇ d R A C ⁇ ⁇ 1 + K C ⁇ d C c 2 , OTT ⁇ ⁇ 4 ⁇ c 2 , OTT ⁇ ⁇ 1 ⁇ c 1 , OTT ⁇ ⁇ 0 A LS ⁇ ⁇ 1 + K Ls ⁇ d Ls A RS ⁇ ⁇ 1 + K Rs ⁇ d Rs ] ⁇ m + ⁇ [ P L ⁇ ⁇ 0 ⁇ d new ⁇ ⁇ 0 ⁇ ( m ) + P L ⁇ ⁇ 1 ⁇ d new ⁇ ⁇ 1 ⁇ ( m ) + ... P R ⁇ ⁇ 0 ⁇ d new ⁇ ⁇ 0 ⁇ ( m ) + P R ⁇ ⁇ 1 ⁇ d new ⁇ ⁇ 1 ⁇ ( m ) + ... P C ⁇
- a filter coefficient generating process uses the same process explained in the first method except that ‘A’ is changed into ‘A+Kd’.
- a ninth method of generating source mapping information using CLD, ICC and/or decorrelators is to generate an additionally decorrelated value by applying a frequency domain decorrelator to an output of the related art decorrelator in case of using the related art decorrelator. Hence, it is able to generate source mapping information with a small operational quantity by overcoming the limitation of the frequency domain decorrelator.
- the output value can be processed on a time domain, a frequency domain, a QMF domain, a hybrid domain, or the like. If the output value is processed on a domain different from a currently processed domain, it can be converted by domain conversion. It is able to use the same ‘d for d_L, d_R, d_C, d_Ls, and d_Rs.
- Math Figure 10 can be expressed in a very simple manner.
- Math Figure 10 is applied to Math Figure 1, Math Figure 1 can be expressed as Math Figure 11.
- rendering information HM_L is a value resulting from combining spatial information and filter information to generate a surround signal Lo with an input m.
- rendering information HM_R is a value resulting from combining spatial information and filter information to generate a surround signal Ro with an input m.
- ‘d(m)’ is a decorrelator output value generated by transferring a decorrelator output value on an arbitrary domain to a value on a current domain or a decorrelator output value generated by being processed on a current domain.
- Rendering information HMD_L is a value indicating an extent of the decorrelator output value d(m) that is added to ‘Lo’ in rendering the d(m), and also a value resulting from combining spatial information and filter information together.
- Rendering information HMD_R is a value indicating an extent of the decorrelator output value d(m) that is added to ‘Ro’ in rendering the d(m).
- the present invention proposes a method of generating a surround signal by rendering the rendering information generated by combining spatial information and filter information (e.g., HRTF filter coefficient) to a downmix signal and a decorrelated downmix signal.
- the rendering process can be executed regardless of domains. If ‘d(m)’ is expressed as ‘d*m’ (product operator) being executed on a frequency domain, Math Figure 11 can be expressed as Math Figure 12.
- FIG. 6 and FIG. 7 are detailed block diagrams of a rendering unit for a stereo downmix signal according to one embodiment of the present invention.
- the rendering unit 900 includes a rendering unit-A 910 and a rendering unit-B 920 .
- the spatial information converting unit 1000 generates rendering information for left and right channels of the downmix signal.
- the rendering unit-A 910 generates a surround signal by rendering the rendering information for the left channel of the downmix signal to the left channel of the downmix signal.
- the rendering unit-B 920 generates a surround signal by rendering the rendering information for the right channel of the downmix signal to the right channel of the downmix signal.
- the names of the channels are just exemplary, which does not put limitation on the present invention.
- the rendering information can include rendering information delivered to a same channel and rendering information delivered to another channel.
- the spatial information converting unit 1000 is able to generate rendering information HL_L and HL_R inputted to the rendering unit for the left channel of the downmix signal, in which rendering information HL_L is delivered to a left output corresponding to the same channel and the rendering information HL_R is delivered to a right output corresponding to the another channel.
- the spatial information converting unit 1000 is able to generate rendering information HR_R and HR_L inputted to the rendering unit for the right channel of the downmix signal, in which the rendering information HR_R is delivered to a right output corresponding to the same channel and the rendering information HR_L is delivered to a left output corresponding to the another channel.
- the rendering unit 900 includes a rendering unit- 1 A 911 , a rendering unit- 2 A 912 , a rendering unit- 1 B 921 , and a rendering unit- 2 B 922 .
- the rendering unit 900 receives a stereo downmix signal and rendering information from the spatial information converting unit 1000 . Subsequently, the rendering unit 900 generates a surround signal by rendering the rendering information to the stereo downmix signal.
- the rendering unit- 1 A 911 performs rendering by using rendering information HL_L delivered to a same channel among rendering information for a left channel of a downmix signal.
- the rendering unit- 2 A 912 performs rendering by using rendering information HL_R delivered to a another channel among rendering information for a left channel of a downmix signal.
- the rendering unit- 1 B 921 performs rendering by using rendering information HR_R delivered to a same channel among rendering information for a right channel of a downmix signal.
- the rendering unit- 2 B 922 performs rendering by using rendering information HR_L delivered to another channel among rendering information for a right channel of a downmix signal.
- the rendering information delivered to another channel is named ‘cross-rendering information’
- the cross-rendering information HL_R or HR_L is applied to a same channel and then added to another channel by an adder.
- the cross-rendering information HL_R and/or HR_L can be zero. If the cross-rendering information HL_R and/or HR_L is zero, it means that no contribution is made to the corresponding path.
- a downmix signal is a stereo signal
- the downmix signal defined as ‘x’, source mapping information generated by using spatial information defined as ‘D’, prototype filter information defined as ‘G’, a multi-channel signal defined as ‘p’ and a surround signal defined as ‘y’ can be represented by matrixes shown in Math Figure 13.
- the multi-channel signal p as shown in Math Figure 14, can be expressed as a product between the source mapping information D generated by using the spatial information and the downmix signal x.
- the surround signal y as shown in Math Figure 15, can be generated by rendering the prototype filter information G to the multi-channel signal p.
- Math Figure 14 if Math Figure 14 is inserted in the p, it can be generated as Math Figure 16.
- the surround signal y and the downmix signal x can have a relation of Math Figure 17.
- the downmix signal x is multiplied by the rendering information H to generate the surround signal y.
- the rendering information H can be expressed as Math Figure 18.
- FIG. 8 and FIG. 9 are detailed block diagrams of a rendering unit for a mono downmix signal according to one embodiment of the present invention.
- the rendering unit 900 includes a rendering unit-A 930 and a rendering unit-B 940 .
- the spatial information converting unit 1000 If a downmix signal is a mono signal, the spatial information converting unit 1000 generates rendering information HM_L and HM_R, in which the rendering information HM_L is used in rendering the mono signal to a left channel and the rendering information HM_R is used in rendering the mono signal to a right channel.
- the rendering unit-A 930 applies the rendering information HM_L to the mono downmix signal to generate a surround signal of the left channel.
- the rendering unit-B 940 applies the rendering information HM_R to the mono downmix signal to generate a surround signal of the right channel.
- the rendering unit 900 in the drawing does not use a decorrelator. Yet, if the rendering unit-A 930 and the rendering unit-B 940 performs rendering by using the rendering information Hmoverall_R and Hmoverall_L defined in Math Figure 12, respectively, it is able to obtain the outputs to which the decorrelator is applied, respectively.
- the first method is that instead of using rendering information for a surround effect, a value used for a stereo output is used. In this case, it is able to obtain a stereo signal by modifying only the rendering information in the structure shown in FIG. 3 .
- the second method is that in a decoding process for generating a multi-channel signal by using a downmix signal and spatial information, it is able to obtain a stereo signal by performing the decoding process to only a corresponding step to obtain a specific channel number.
- the rendering unit 900 corresponds to a case in which a decorrelated signal is represented as one, i.e., Math Figure 11.
- the rendering unit 900 includes a rendering unit- 1 A 931 , a rendering unit- 2 A 932 , a rendering unit- 1 B 941 , and a rendering unit- 2 B 942 .
- the rendering unit 900 is similar to the rendering unit for the stereo downmix signal except that the rendering unit 900 includes the rendering units 941 and 942 for a decorrelated signal.
- the rendering unit- 1 A 931 generates a signal to be delivered to a same channel by applying the rendering information HM_L to a mono downmix signal.
- the rendering unit- 2 A 932 generates a signal to be delivered to another channel by applying the rendering information HM_R to the mono downmix signal.
- the rendering unit- 1 B 941 generates a signal to be delivered to a same channel by applying the rendering information HMD_R to a decorrelated signal.
- the rendering unit- 2 B 942 generates a signal to be delivered to another channel by applying the rendering information HMD_L to the decorrelated signal.
- a downmix signal is a mono signal
- a downmix signal defined as x source channel information defined as D
- prototype filter information defined as G prototype filter information defined as G
- p multi-channel signal defined as p
- a surround signal defined as y can be represented by matrixes shown in Math Figure 19.
- the relation between the matrixes is similar to that of the case that the downmix signal is the stereo signal. So its details are omitted.
- the source mapping information described with reference to FIG. 4 and FIG. 5 and the rendering information generated by using the source mapping information have values differing per frequency band, parameter band, and/or transmitted timeslot.
- a value of the source mapping information and/or the rendering information has a considerably big difference between neighbor bands or between boundary timeslots, distortion may take place in the rendering process.
- a smoothing process on a frequency and/or time domain is needed.
- Another smoothing method suitable for the rendering is usable as well as the frequency domain smoothing and/or the time domain smoothing. And, it is able to use a value resulting from multiplying the source mapping information or the rendering information by a specific gain.
- FIG. 10 and FIG. 11 are block diagrams of a smoothing unit and an expanding unit according to one embodiment of the present invention.
- a smoothing method according to the present invention is applicable to rendering information and/or source mapping information. Yet, the smoothing method is applicable to other type information. In the following description, smoothing on a frequency domain is described. Yet, the present invention includes time domain smoothing as well as the frequency domain smoothing.
- the smoothing unit 1042 is capable of performing smoothing on rendering information and/or source mapping information. A detailed example of a position of the smoothing occurrence will be described with reference to FIGS. 18 to 20 later.
- the smoothing unit 1042 can be configured with an expanding unit 1043 , in which the rendering information and/or source mapping information can be expanded into a wider range, for example filter band, than that of a parameter band.
- the source mapping information can be expanded to a frequency resolution (e.g., filter band) corresponding to filter information to be multiplied by the filter information (e.g., HRTF filter coefficient).
- the smoothing according to the present invention is executed prior to or together with the expansion.
- the smoothing used together with the expansion can employ one of the methods shown in FIGS. 12 to 16 .
- FIG. 12 is a graph to explain a first smoothing method according to one embodiment of the present invention.
- a first smoothing method uses a value having the same size as spatial information in each parameter band. In this case, it is able to achieve a smoothing effect by using a suitable smoothing function.
- FIG. 13 is a graph to explain a second smoothing method according to one embodiment of the present invention.
- a second smoothing method is to obtain a smoothing effect by connecting representative positions of parameter band.
- the representative position is a right center of each of the parameter bands, a central position proportional to a log scale, a bark scale, or the like, a lowest frequency value, or a position previously determined by a different method.
- FIG. 14 is a graph to explain a third smoothing method according to one embodiment of the present invention.
- a third smoothing method is to perform smoothing in a form of a curve or straight line smoothly connecting boundaries of parameters.
- the third smoothing method uses a preset boundary smoothing curve or low pass filtering by the first order or higher IIR filter or FIR filter.
- FIG. 15 is a graph to explain a fourth smoothing method according to one embodiment of the present invention.
- a fourth smoothing method is to achieve a smoothing effect by adding a signal such as a random noise to a spatial information contour.
- a value differing in channel or band is usable as the random noise.
- the fourth smoothing method is able to achieve an inter-channel decorrelation effect as well as a smoothing effect on a frequency domain.
- FIG. 16 is a graph to explain a fifth smoothing method according to one embodiment of the present invention.
- a fifth smoothing method is to use a combination of the second to fourth smoothing methods. For instance, after the representative positions of the respective parameter bands have been connected, the random noise is added and low path filtering is then applied. In doing so, the sequence can be modified.
- the fifth smoothing method minimizes discontinuous points on a frequency domain and an inter-channel decorrelation effect can be enhanced.
- a total of powers for spatial information values (e.g., CLD values) on the respective frequency domains per channel should be uniform as a constant.
- power normalization should be performed. For instance, if a downmix signal is a mono signal, level values of the respective channels should meet the relation of Math Figure 20.
- FIG. 17 is a diagram to explain prototype filter information per channel.
- a signal having passed through GL_L filter for a left channel source is sent to a left output, whereas a signal having passed through GL_R filter is sent to a right output.
- a left final output e.g., Lo
- a right final output e.g., Ro
- the rendered left/right channel outputs can be expressed as Math Figure 21.
- Lo L*GL — L+C*GC — L+R*GR — L+Ls*GLs — L+Rs*GRs — L
- the rendered left/right channel outputs can be generated by using the L, R, C, Ls, and Rs generated by decoding the downmix signal into the multi-channel signal using the spatial information. And, the present invention is able to generate the rendered left/right channel outputs using the rendering information without generating the L, R, C, Ls, and Rs, in which the rendering information is generated by using the spatial information and the filter information.
- FIGS. 18 to 20 A process for generating rendering information using spatial information is explained with reference to FIGS. 18 to 20 as follows.
- FIG. 18 is a block diagram for a first method of generating rendering information in a spatial information converting unit 900 according to one embodiment of the present invention.
- the spatial information converting unit 900 includes the source mapping unit 1010 , the sub-rendering information generating unit 1020 , the integrating unit 1030 , the processing unit 1040 , and the domain converting unit 1050 .
- the spatial information converting unit 900 has the same configuration shown in FIG. 3 .
- the sub-rendering information generating unit 1020 includes at least one or more sub-rendering information generating units (1 st sub-rendering information generating unit to N th sub-rendering information generating unit).
- the sub-rendering information generating unit 1020 generates sub-rendering information by using filter information and source mapping information.
- the first sub-rendering information generating unit is able to generate sub-rendering information corresponding to a left channel on a multi-channel.
- the sub-rendering information can be represented as Math Figure 22 using the source mapping information D_L and the converted filter information GL_L′ and GL_R′
- the D_L is a value generated by using the spatial information in the source mapping unit 1010 . Yet, a process for generating the D_L can follow the tree structure.
- the second sub-rendering information generating unit is able to generate sub-rendering information FR_L and FR_R corresponding to a right channel on the multi-channel.
- the N th sub-rendering information generating unit is able to generate sub-rendering information FRs_L and FRs_R corresponding to a right surround channel on the multi-channel.
- the first sub-rendering information generating unit is able to generate sub-rendering information corresponding to the left channel on the multi-channel.
- the sub-rendering information can be represented as Math Figure 23 by using the source mapping information D_L 1 and D_L 2 .
- ‘L’ indicates a position of the multi-channel
- ‘R’ indicates an output channel of a surround signal
- ‘1’ indicates a channel of the downmix signal.
- the FL_R 1 indicates the sub-rendering information used in generating the right output channel of the surround signal from the left channel of the downmix signal.
- the D_L 1 and the D_L 2 are values generated by using the spatial information in the source mapping unit 1010 .
- a downmix signal is a stereo signal, it is able to generate a plurality of sub-rendering informations from at least one sub-rendering information generating unit in the same manner of the case that the downmix signal is the mono signal.
- the types of the sub-rendering informations generated by a plurality of the sub-rendering information generating units are exemplary, which does not put limitation on the present invention.
- the sub-rendering information generated by the sub-rendering information generating unit 1020 is transferred to the rendering unit 900 via the integrating unit 1030 , the processing unit 1040 , and the domain converting unit 1050 .
- the integrating unit 1030 integrates the sub-rendering informations generated per channel into rendering information (e.g., HL_L, HL_R, HR_L, HR_R) for a rendering process.
- rendering information e.g., HL_L, HL_R, HR_L, HR_R
- An integrating process in the integrating unit 1030 is explained for a case of a mono signal and a case of a stereo signal as follows.
- HM — L FL — L+FR — L+FC — L+FLs — L+FRs — L+FLFE — L
- rendering information can be expressed as Math Figure 25.
- the processing unit 1040 includes an interpolating unit 1041 and/or a smoothing unit 1042 and performs interpolation and/or smoothing for the rendering information.
- the interpolation and/or smoothing can be executed on a time domain, a frequency domain, or a QMF domain.
- the time domain is taken as an example, which does not put limitation on the present invention.
- the interpolation is performed to obtain rendering information non-existing between the rendering informations if the transmitted rendering information has a wide interval on the time domain. For instance, assuming that rendering informations exist in an n th timeslot and an (n+k) th timeslot (k>1), respectively, it is able to perform linear interpolation on a not-transmitted timeslot by using the generated rendering informations (e.g., HL_L, HR_L, HL_R, HR_R).
- the generated rendering informations e.g., HL_L, HR_L, HL_R, HR_R.
- the rendering information generated from the interpolation is explained with reference to a case that a downmix signal is a mono signal and a case that the downmix signal is a stereo signal.
- the interpolated rendering information can be expressed as Math Figure 26.
- HM — L ( n+j ) HM — L ( n )*(1 ⁇ a )+ HM — L ( n+k )* a
- HM — R ( n+j ) HM — R ( n )*(1 ⁇ a )+ HM — R ( n+k )* a Math Figure 26
- the interpolated rendering information can be expressed as Math Figure 27.
- HL — L ( n+j ) HL — L ( n )*(1 ⁇ a )+ HL — L ( n+k )* a
- HL — R ( n+j ) HL — R ( n )*(1 ⁇ a )+ HL — R ( n+k )* a
- the smoothing unit 1042 executes smoothing to prevent a problem of distortion due to an occurrence of a discontinuous point.
- the smoothing on the time domain can be carried out using the smoothing method described with reference to FIGS. 12 to 16 .
- the smoothing can be performed together with expansion. And, the smoothing may differ according to its applied position. If a downmix signal is a mono signal, the time domain smoothing can be represented as Math Figure 29.
- HM — L ( n )′ HM — L ( n )* b+HM — L ( n ⁇ 1)′*(1 ⁇ b )
- the smoothing can be executed by the I-pol HR filter type performed in a manner of multiplying the rendering information HM_L(n ⁇ 1) or HM_R(n ⁇ 1) smoothed in a previous timeslot n ⁇ 1 by (1 ⁇ b), multiplying the rendering information HM_L(n) or HM)R(n) generated in a current timeslot n by b, and adding the two multiplications together.
- ‘b’ is a constant for 0 ⁇ b ⁇ 1. If ‘b’ gets smaller, a smoothing effect becomes greater. If ‘b’ gets bigger, a smoothing effect becomes smaller. And, the rest of the filters can be applied in the same manner.
- the interpolation and the smoothing can be represented as one expression shown in Math Figure 30 by using Math Figure 29 for the time domain smoothing.
- interpolation is performed by the interpolating unit 1041 and/or if the smoothing is performed by the smoothing unit 1042 , rendering information having an energy value different from that of prototype rendering information may be obtained. To prevent this problem, energy normalization may be executed in addition.
- the domain converting unit 1050 performs domain conversion on the rendering information for a domain for executing the rendering. If the domain for executing the rendering is identical to the domain of rendering information, the domain conversion may not be executed. Thereafter, the domain-converted rendering information is transferred to the rendering unit 900 .
- FIG. 19 is a block diagram for a second method of generating rendering information in a spatial information converting unit according to one embodiment of the present invention.
- the second method is similar to the first method in that a spatial information converting unit 1000 includes a source mapping unit 1010 , a sub-rendering information generating unit 1020 , an integrating unit 1030 , a processing unit 1040 , and a domain converting unit 1050 and in that the sub-rendering information generating unit 1020 includes at least one sub-rendering information generating unit.
- the second method of generating the rendering information differs from the first method in a position of the processing unit 1040 . So, interpolation and/or smoothing can be performed per channel on sub-rendering informations (e.g., FL_L and FL_R in case of mono signal or FL_L 1 , FL_L 2 , FL_R 1 , FL_R 2 in case of stereo signal) generated per channel in the sub-rendering information generating unit 1020 .
- sub-rendering informations e.g., FL_L and FL_R in case of mono signal or FL_L 1 , FL_L 2 , FL_R 1 , FL_R 2 in case of stereo signal
- the integrating unit 1030 integrates the interpolated and/or smoothed sub-rendering informations into rendering information.
- the generated rendering information is transferred to the rendering unit 900 via the domain converting unit 1050 .
- FIG. 20 is a block diagram for a third method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention.
- the third method is similar to the first or second method in that a spatial information converting unit 1000 includes a source mapping unit 1010 , a sub-rendering information generating unit 1020 , an integrating unit 1030 , a processing unit 1040 , and a domain converting unit 1050 and in that the sub-rendering information generating unit 1020 includes at least one sub-rendering information generating unit.
- the third method of generating the rendering information differs from the first or second method in that the processing unit 1040 is located next to the source mapping unit 1010 . So, interpolation and/or smoothing can be performed per channel on source mapping information generated by using spatial information in the source mapping unit 1010 .
- the sub-rendering information generating unit 1020 generates sub-rendering information by using the interpolated and/or smoothed source mapping information and filter information.
- the sub-rendering information is integrated into rendering information in the integrating unit 1030 . And, the generated rendering information is transferred to the rendering unit 900 via the domain converting unit 1050 .
- FIG. 21 is a diagram to explain a method of generating a surround signal in a rendering unit according to one embodiment of the present invention.
- FIG. 21 shows a rendering process executed on a DFT domain. Yet, the rendering process can be implemented on a different domain in a similar manner as well.
- FIG. 21 shows a case that an input signal is a mono downmix signal. Yet, FIG. 21 is applicable to other input channels including a stereo downmix signal and the like in the same manner.
- a mono downmix signal on a time domain preferentially executes windowing having an overlap interval OL in the domain converting unit.
- FIG. 21 shows a case that 50% overlap is used. Yet, the present invention includes cases of using other overlaps.
- a window function for executing the windowing can employ a function having a good frequency selectivity on a DFT domain by being seamlessly connected without discontinuity on a time domain.
- a sine square window function can be used as the window function.
- FIG. 20 shows that a block-k downmix signal is domain-converted into a DFT domain.
- the domain-converted downmix signal is rendered by a rendering filter that uses rendering information.
- the rendering process can be represented as a product of a downmix signal and rendering information.
- the rendered downmix signal undergoes IDFT (Inverse Discrete Fourier Transform) in the inverse domain converting unit and is then overlapped with the downmix signal (block k ⁇ 1 in FIG. 20 ) previously executed with a delay of a length OL to generate a surround signal.
- IDFT Inverse Discrete Fourier Transform
- Interpolation can be performed on each block undergoing the rendering process.
- the interpolating method is explained as follows.
- FIG. 22 is a diagram for a first interpolating method according to one embodiment of the present invention.
- Interpolation according to the present invention can be executed on various positions.
- the interpolation can be executed on various positions in the spatial information converting unit shown in FIGS. 18 to 20 or can be executed in the rendering unit.
- Spatial information, source mapping information, filter information and the like can be used as the values to be interpolated.
- the spatial information is exemplarily used for description.
- the present invention is not limited to the spatial information.
- the interpolation is executed after or together with expansion to a wider band.
- spatial information transferred from an encoding apparatus c an be transferred from a random position instead of being transmitted each timeslot.
- One spatial frame is able to carry a plurality of spatial information sets (e.g., parameter sets n and n+1 in FIG. 22 ).
- one spatial frame is able to carry a single new spatial information set. So, interpolation is carried out for a not-transmitted timeslot using values of a neighboring transmitted spatial information set. An interval between windows for executing rendering does not always match a timeslot. So, an interpolated value at a center of the rendering windows (K ⁇ 1, K, K+1, K+2, etc.), as shown in FIG. 22 , is found to use.
- interpolation is carried out between timeslots where a spatial information set exists
- the present invention is not limited to the interpolating method. For instance, interpolation is not carried out on a timeslot where a spatial information set does not exist. Instead, a previous or preset value can be used.
- FIG. 23 is a diagram for a second interpolating method according to one embodiment of the present invention.
- a second interpolating method has a structure that an interval using a previous value, an interval using a preset default value and the like are combined.
- interpolation can be performed by using at least one of a method of maintaining a previous value, a method of using a preset default value, and a method of executing linear interpolation in an interval of one spatial frame.
- distortion may take place.
- block switching for preventing the distortion is explained.
- FIG. 24 is a diagram for a block switching method according to one embodiment of the present invention.
- a window length is greater than a timeslot length
- at least two spatial information sets e.g., parameter sets n and n+1 in FIG. 24
- each of the spatial information sets should be applied to a different timeslot.
- distortion may take place. Namely, distortion attributed to time resolution shortage according to a window length can take place.
- a switching method of varying a window size to fit resolution of a timeslot can be used. For instance, a window size, as shown in (b) of FIG. 24 , can be switched to a shorter-sized window for an interval requesting a high resolution. In this case, at a beginning and an ending portion of switched windows, connecting windows is used to prevent seams from occurring on a time domain of the switched windows.
- the window length can be decided by using spatial information in a decoding apparatus instead of being transferred as separate additional information. For instance, a window length can be determined by using an interval of a timeslot for updating spatial information. Namely, if the interval for updating the spatial information is narrow, a window function of short length is used. If the interval for updating the spatial information is wide, a window function of long length is used. In this case, by using a variable length window in rendering, it is advantageous not to use bits for sending window length information separately. Two types of window length are shown in (b) of FIG. 24 . Yet, windows having various lengths can be used according to transmission frequency and relations of spatial information. The decided window length information is applicable to various steps for generating a surround signal, which is explained in the following description.
- FIG. 25 is a block diagram for a position to which a window length decided by a window length deciding unit is applied according to one embodiment of the present invention.
- a window length deciding unit 1400 is able to decide a window length by using spatial information.
- Information for the decided window length is applicable to a source mapping unit 1010 , an integrating unit 1030 , a processing unit 1040 , domain converting units 1050 and 1100 , and a inverse domain converting unit 1300 .
- FIG. 25 shows a case that a stereo downmix signal is used. Yet, the present invention is not limited to the stereo downmix signal only. As mentioned in the foregoing description, even if a window length is shortened, a length of zero padding decided according to a filter tab number is not adjustable. So, a solution for the problem is explained in the following description.
- FIG. 26 is a diagram for filters having various lengths used in processing an audio signal according to one embodiment of the present invention.
- a solution for the problem is to reduce the length of the zero padding by restricting a length of a filter tab.
- a method of reducing the length of the zero padding can be achieved by truncating a rear portion of a response (e.g., a diffusing interval corresponding to reverberation). In this case, a rendering process may be less accurate than a case of not truncating the rear portion of the filter response.
- filter coefficient values on a time domain are very small to mainly affect reverberation. So, a sound quality is not considerably affected by the truncating.
- the four kinds of the filters are usable on a DFT domain, which does not put limitation on the present invention.
- a filter-N indicates a filter having a long filter length FL and a length 2*OL of a long zero padding of which filter tab number is not restricted.
- a filter-N 2 indicates a filter having a zero padding length 2*OL shorter than that of the filter-N 1 by restricting a tab number of filter with the same filter length FL.
- a filter-N 3 indicates a filter having a long zero padding length 2*OL by not restricting a tab number of filter with a filter length FL shorter than that of the filter-N 1 .
- a filter-N 4 indicates a filter having a window length FL shorter than that of the filter-N 1 with a short zero padding length 2*OL by restricting a tab number of filter.
- FIG. 27 is a diagram for a method of processing an audio signal dividedly by using a plurality of subfilters according to one embodiment of the present invention.
- One filter may be divided into subfilters having filter coefficients differing from each other.
- a method of adding results of the processing can be used.
- the method provides function for processing dividedly the audio signal by a predetermined length unit. For instance, since the rear portion of the filter response is not considerably varied per HRTF corresponding to each channel, it is able to perform the rendering by extracting a coefficient common to a plurality of windows.
- a case of execution on a DFT domain is described. Yet, the present invention is not limited to the DFT domain.
- a plurality of the sub-areas can be processed by a plurality of subfilters (filter-A and filter-B) having filter coefficients differing from each other.
- an output processed by the filter-A and an output processed by the filter-B are combined together.
- IDFT Inverse Discrete Fourier Transform
- the generated signals are added together.
- a position, to which the output processed by the filter-B is added is time-delayed by FL more than a position of the output processed by the filter-A.
- the signal processed by a plurality of the subfilters brings the same effect of the case that the signal is processed by a single filter.
- the present invention includes a method of rendering the output processed by the filter-B to a downmix signal directly.
- it is able to render the output to the downmix signal by using coefficients extracting from spatial information, the spatial information in part or without using the spatial information.
- the method is characterized in that a filter having a long tab number can be applied dividedly and that a rear portion of the filter having small energy is applicable without conversion using spatial information. In this case, if conversion using spatial information is not applied, a different filter is not applied to each processed window. So, it is unnecessary to apply the same scheme as the block switching.
- FIG. 26 shows that the filter is divided into two areas. Yet, the present invention is able to divide the filter into a plurality of areas.
- FIG. 28 is a block diagram for a method of rendering partition rendering information generated by a plurality of subfilters to a mono downmix signal according to one embodiment of the present invention.
- FIG. 28 relates to one rendering coefficient. The method can be executed per rendering coefficient.
- the filter-A information of FIG. 27 corresponds to first partition rendering information HM_L_A and the filter-B information of FIG. 27 corresponds to second partition rendering information HM_L_B.
- FIG. 28 shows an embodiment of partition into two subfilters. Yet, the present invention is not limited to the two subfilters.
- the two subfilters can be obtained via a splitting unit 1500 using the rendering information HM_L generated in the spatial information generating unit 1000 .
- the two subfilters can be obtained using prototype HRTF information or information decided according to a user's selection.
- the information decided according to a user's selection may include spatial information selected according to a user's taste for example.
- HM_L_A is the rendering information based on the received spatial information and, HM_L_B may be the rendering information for providing a 3-dimensional effect commonly applied to signals.
- the processing with a plurality of the subfilters is applicable to a time domain and a QMF domain as well as the DFT domain.
- the coefficient values split by the filter-A and the filter-B are applied to the downmix signal by time or QMF domain rendering and are then added to generate a final signal.
- the rendering unit 900 includes a first partition rendering unit 950 and a second partition rendering unit 960 .
- the first partition rendering unit 950 performs a rendering process using HM_L_A
- the second partition rendering unit 960 performs a rendering process using HM_L_B.
- FIG. 28 shows an example of a mono downmix signal.
- a portion corresponding to the filter-B is applied not to the decorrelator but to the mono downmix signal directly.
- FIG. 29 is a block diagram for a method of rendering partition rendering information generated using a plurality of subfilters to a stereo downmix signal according to one embodiment of the present invention.
- a partition rendering process shown in FIG. 29 is similar to that of FIG. 28 in that two subfilters are obtained in a splitter 1500 by using rendering information generated by the spatial information converting unit 1000 , prototype HRTF filter information or user decision information.
- the difference from FIG. 28 lies in that a partition rendering process corresponding to the filter-B is commonly applied to LUR signals.
- the splitter 1500 generates first partition rendering information corresponding to filter-A information, second partition rendering information, and third partition rendering information corresponding to filter-B information.
- the third partition rendering information can be generated by using filter information or spatial information commonly applicable to the LIR signals.
- a rendering unit 900 includes a first partition rendering unit 970 , a second partition rendering unit 980 , and a third partition rendering unit 990 .
- the third partition rendering information generates is applied to a sum signal of the L/R signals in the third partition rendering unit 990 to generate one output signal.
- the output signal is added to the L/R output signals, which are independently rendered by a filter-A 1 and a filter-A 2 in the first and second partition rendering units 970 and 980 , respectively, to generate surround signals.
- the output signal of the third partition rendering unit 990 can be added after an appropriate delay.
- FIG. 29 an expression of cross rendering information applied to another channel from L/R inputs is omitted for convenience of explanation.
- FIG. 30 is a block diagram for a first domain converting method of a downmix signal according to one embodiment of the present invention.
- the rendering process executed on the DFT domain has been described so far. As mentioned in the foregoing description, the rendering process is executable on other domains as well as the DFT domain. Yet, FIG. 30 shows the rendering process executed on the DFT domain.
- a domain converting unit 1100 includes a QMF filter and a DFT filter.
- An inverse domain converting unit 1300 includes an IDFT filter and an IQMF filter.
- FIG. 30 relates to a mono downmix signal, which does not put limitation on the present invention.
- a time domain downmix signal of p samples passes through a QMF filter to generate P sub-band samples. W samples are recollected per band. After windowing is performed on the recollected samples, zero padding is performed. M-point DFT (FFT) is then executed. In this case, the DFT enables a processing by the aforesaid type windowing.
- a value connecting the M/2 frequency domain values per band obtained by the M-point DFT to P bands can be regarded as an approximate value of a frequency spectrum obtained by M/2*P-point DFT. So, a filter coefficient represented on a M/2*P-point DFT domain is multiplied by the frequency spectrum to bring the same effect of the rendering process on the DFT domain.
- the signal having passed through the QMF filter has leakage, e.g., aliasing between neighboring bands.
- a value corresponding to a neighbor band smears in a current band and a portion of a value existing in the current band is shifted to the neighbor band.
- QMF integration is executed, an original signal can be recovered due to QMF characteristics.
- a filtering process is performed on the signal of the corresponding band as the case in the present invention, the signal is distorted by the leakage.
- a process for recovering an original signal can be added in a manner of having a signal pass through a leakage minimizing butterfly B prior to performing DFT per band after QMF in the domain converting unit 100 and performing a reversing process V after IDFT in the inverse domain converting unit 1300 .
- DFT can be performed on a QMF pass signal for prototype filter information instead of executing M/2*P-point DFT in the beginning.
- delay and data spreading due to QMF filter may exist.
- FIG. 31 is a block diagram for a second domain converting method of a downmix signal according to one embodiment of the present invention.
- FIG. 31 shows a rendering process performed on a QMF domain.
- a domain converting unit 1100 includes a QMF domain converting unit and an inverse domain converting unit 1300 includes an IQMF domain converting unit.
- a configuration shown in FIG. 31 is equal to that of the case of using DFT only except that the domain converting unit is a QMF filter.
- the QMF is referred to as including a QMF and a hybrid QMF having the same bandwidth.
- the difference from the case of using DFT only lies in that the generation of the rendering information is performed on the QMF domain and that the rendering process is represented as a convolution instead of the product on the DFT domain, since the rendering process performed by a renderer-M 3012 is executed on the QMF domain.
- a filter coefficient can be represented as a set of filter coefficients having different features (coefficients) for the B bands.
- a filter tab number becomes a first order (i.e., multiplied by a constant)
- a rendering process on a DFT domain having B frequency spectrums and an operational process are matched.
- Math Figure 31 represents a rendering process executed in one QMF band (b) for one path for performing the rendering process using rendering information HM_L.
- k indicates a time order in QMF band, i.e., a timeslot unit.
- the rendering process executed on the QMF domain is advantageous in that, if spatial information transmitted is a value applicable to the QMF domain, application of corresponding data is most facilitated and that distortion in the course of application can be minimized.
- QMF domain conversion in the prototype filter information (e.g., prototype filter coefficient) converting process a considerable operational quantity is required for a process of applying the converted value.
- the operational quantity can be minimized by the method of parameterizing the HRTF coefficient in the filter information converting process.
- the signal processing method and apparatus of the present invention uses spatial information provided by an encoder to generate surround signals by using HRTF filter information or filter information according to a user in a decoding apparatus in capable of generating multi-channels. And, the present invention is usefully applicable to various kinds of decoders capable of reproducing stereo signals only.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Algebra (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- General Physics & Mathematics (AREA)
- Stereophonic System (AREA)
- Image Generation (AREA)
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
- Television Systems (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
An apparatus for processing a media signal and method thereof are disclosed, by which the media signal can be converted to a surround signal by using spatial information of the media signal. The present invention provides a method of processing a signal, the method comprising of generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources; generating at least one rendering information by using the source mapping information and filter information having a surround effect; and performing smoothing by using neighbor rendering information of the at least one rendering information.
Description
- The present invention relates to an apparatus for processing a media signal and method thereof, and more particularly to an apparatus for generating a surround signal by using spatial information of the media signal and method thereof.
- Generally, various kinds of apparatuses and methods have been widely used to generate a multi-channel media signal by using spatial information for the multi-channel media signal and a downmix signal, in which the downmix signal is generated by downmixing the multi-channel media signal into mono or stereo signal.
- However, the above methods and apparatuses are not usable in environments unsuitable for generating a multi-channel signal. For instance, they are not usable for a device capable of generating only a stereo signal. In other words, there exists no method or apparatus for generating a surround signal, in which the surround signal has multi-channel features in the environment incapable of generating a multi-channel signal by using spatial information of the multi-channel signal.
- So, since there exists no method or apparatus for generating a surround signal in a device capable of generating only a mono or stereo signal, it is difficult to process the media signal efficiently.
- Accordingly, the present invention is directed to an apparatus for processing a media signal and method thereof that substantially obviate one or more of the problems due to limitations and disadvantages of the related art.
- An object of the present invention is to provide an apparatus for processing a media signal and method thereof, by which the media signal can be converted to a surround signal by using spatial information for the media signal.
- Additional features and advantages of the invention will be set forth in a description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
- To achieve these and other advantages and in accordance with the purpose of the present invention, a method of processing a signal according to the present invention includes of: generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources; generating sub-rendering information by applying filter information giving a surround effect to the source mapping information per the source; generating rendering information for generating a surround signal by integrating at least one of the sub-rendering information; and generating the surround signal by applying the rendering information to a downmix signal generated by downmixing the multi-sources.
- To further achieve these and other advantages and in accordance with the purpose of the present invention, an apparatus for processing a signal includes a source map ping unit generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources; a sub-rendering information generating unit generating sub-rendering information by applying filter information having a surround effect to the source mapping information per the source; an integrating unit generating rendering information for generating a surround signal by integrating the at least one of the sub-rendering information; and a rendering unit generating the surround signal by applying the rendering information to a downmix signal generated by downmixing the multi-sources.
- It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
- A signal processing apparatus and method according to the present invention enable a decoder, which receives a bitstream including a downmix signal generated by downmixing a multi-channel signal and spatial information of the multi-channel signal, to generate a signal having a surround effect in environments in incapable of recovering the multi-channel signal.
- The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
- In the drawings:
-
FIG. 1 is a block diagram of an audio signal encoding apparatus and an audio signal decoding apparatus according to one embodiment of the present invention; -
FIG. 2 is a structural diagram of a bitstream of an audio signal according to one embodiment of the present invention; -
FIG. 3 is a detailed block diagram of a spatial information converting unit according to one embodiment of the present invention; -
FIG. 4 andFIG. 5 are block diagrams of channel configurations used for source mapping process according to one embodiment of the present invention; -
FIG. 6 andFIG. 7 are detailed block diagrams of a rendering unit for a stereo downmix signal according to one embodiment of the present invention; -
FIG. 8 andFIG. 9 are detailed block diagrams of a rendering unit for a mono downmix signal according to one embodiment of the present invention; -
FIG. 10 andFIG. 11 are block diagrams of a smoothing unit and an expanding unit according to one embodiment of the present invention; -
FIG. 12 is a graph to explain a first smoothing method according to one embodiment of the present invention; -
FIG. 13 is a graph to explain a second smoothing method according to one embodiment of the present invention; -
FIG. 14 is a graph to explain a third smoothing method according to one embodiment of the present invention; -
FIG. 15 is a graph to explain a fourth smoothing method according to one embodiment of the present invention; -
FIG. 16 is a graph to explain a fifth smoothing method according to one embodiment of the present invention; -
FIG. 17 is a diagram to explain prototype filter information corresponding to each channel; -
FIG. 18 is a block diagram for a first method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention; -
FIG. 19 is a block diagram for a second method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention; -
FIG. 20 is a block diagram for a third method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention; -
FIG. 21 is a diagram to explain a method of generating a surround signal in a rendering unit according to one embodiment of the present invention; -
FIG. 22 is a diagram for a first interpolating method according to one embodiment of the present invention; -
FIG. 23 is a diagram for a second interpolating method according to one embodiment of the present invention; -
FIG. 24 is a diagram for a block switching method according to one embodiment of the present invention; -
FIG. 25 is a block diagram for a position to which a window length decided by a window length deciding unit is applied according to one embodiment of the present invention; -
FIG. 26 is a diagram for filters having various lengths used in processing an audio signal according to one embodiment of the present invention; -
FIG. 27 is a diagram for a method of processing an audio signal dividedly by using a plurality of subfilters according to one embodiment of the present invention; -
FIG. 28 is a block diagram for a method of rendering partition rendering information generated by a plurality of subfilters to a mono downmix signal according to one embodiment of the present invention; -
FIG. 29 is a block diagram for a method of rendering partition rendering information generated by a plurality of subfilters to a stereo downmix signal according to one embodiment of the present invention; -
FIG. 30 is a block diagram for a first domain converting method of a downmix signal according to one embodiment of the present invention; and -
FIG. 31 is a block diagram for a second domain converting method of a downmix signal according to one embodiment of the present invention. - Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.
-
FIG. 1 is a block diagram of an audio signal encoding apparatus and an audio signal decoding apparatus according to one embodiment of the present invention. - Referring to
FIG. 1 , anencoding apparatus 10 includes adownmixing unit 100, a spatialinformation generating unit 200, a downmixsignal encoding unit 300, a spatialinformation encoding unit 400, and amultiplexing unit 500. - If multi-source (X1, X2, . . . , Xn) audio signal is inputted to the
downmixing unit 100, thedownmixing unit 100 downmixes the inputted signal into a downmix signal. In this case, the downmix signal includes mono, stereo and multi-source audio signal. - The source includes a channel and, in convenience, is represented as a channel in the following description. In the present specification, the mono or stereo downmix signal is referred to as a reference. Yet, the present invention is not limited to the mono or stereo downmix signal.
- The
encoding apparatus 10 is able to optionally use an arbitrary downmix signal directly provided from an external environment. - The spatial
information generating unit 200 generates spatial information from a multi-channel audio signal. The spatial information can be generated in the course of a downmixing process. The generated downmix signal and spatial information are encoded by the downmixsignal encoding unit 300 and the spatialinformation encoding unit 400, respectively and are then transferred to themultiplexing unit 500. - In the present invention, ‘spatial information’ means information necessary to generate a multi-channel signal from upmixing a downmix signal by a decoding apparatus, in which the downmix signal is generated by downmixing the multi-channel signal by an encoding apparatus and transferred to the decoding apparatus. The spatial information includes spatial parameters. The spatial parameters include CLD (channel level difference) indicating an energy difference between channels, ICC (inter-channel coherences) indicating a correlation between channels, CPC (channel prediction coefficients) used in generating three channels from two channels, etc.
- In the present invention, ‘downmix signal encoding unit’ or ‘downmix signal decoding unit’ means a codec that encodes or decodes an audio signal instead of spatial information. In the present specification, a downmix audio signal is taken as an example of the audio signal instead of the spatial information. And, the downmix signal encoding or decoding unit may include MP3, AC-3, DTS, or AAC. Moreover, the downmix signal encoding or decoding unit may include a codec of the future as well as the previously developed codec.
- The
multiplexing unit 500 generates a bitstream by multiplexing the downmix signal and the spatial information and then transfers the generated bitstream to thedecoding apparatus 20. Besides, the structure of the bitstream will be explained inFIG. 2 later. - A
decoding apparatus 20 includes a demultiplexing unit 600, a downmixsignal decoding unit 700, a spatial information decoding unit 800, arendering unit 900, and a spatialinformation converting unit 1000. - The demultiplexing unit 600 receives a bitstream and then separates an encoded downmix signal and an encoded spatial information from the bitstream. Subsequently, the downmix
signal decoding unit 700 decodes the encoded downmix signal and the spatial information decoding unit 800 decodes the encoded spatial information. - The spatial
information converting unit 1000 generates rendering information applicable to a downmix signal using the decoded spatial information and filter information. In this case, the rendering information is applied to the downmix signal to generate a surround signal. - For instance, the surround signal is generated in the following manner. First of all, a process for generating a downmix signal from a multi-channel audio signal by the
encoding apparatus 10 can include several steps using an OTT (one-to-two) or TTT (three-to-three) box. In this case, spatial information can be generated from each of the steps. The spatial information is transferred to thedecoding apparatus 20. Thedecoding apparatus 20 then generates a surround signal by converting the spatial information and then rendering the converted spatial information with a downmix signal. Instead of generating a multi-channel signal by upmixing a downmix signal, the present invention relates to a rendering method including the steps of extracting spatial information for each upmixing step and performing a rendering by using the extracted spatial information. For example, HRTF (head-related transfer functions) filtering is usable in the rendering method. - In this case, the spatial information is a value applicable to a hybrid domain as well. So, the rendering can be classified into the following types according to a domain.
- The first type is that the rendering is executed on a hybrid domain by having a downmix signal pass through a hybrid filterbank. In this case, a conversion of domain for spatial information is unnecessary.
- The second type is that the rendering is executed on a time domain. In this case, the second type uses a fact that a HRTF filter is modeled as a FIR (finite inverse response) filter or an IIR (infinite inverse response) filter on a time domain. So, a process for converting spatial information to a filter coefficient of time domain is needed.
- The third type is that the rendering is executed on a different frequency domain. For instance, the rendering is executed on a DFT (discrete Fourier transform) domain. In this case, a process for transforming spatial information into a corresponding domain is necessary. In particular, the third type enables a fast operation by replacing a filtering on a time domain into an operation on a frequency domain.
- In the present invention, filter information is the information for a filter necessary for processing an audio signal and includes a filter coefficient provided to a specific filter. Examples of the filter information are explained as follows. First of all, prototype filter information is original filter information of a specific filter and can be represented as GL_L or the like. Converted filter information indicates a filter coefficient after the prototype filter information has been converted and can be represented as GL_L or the like. Sub-rendering information means the filter information resulting from spatializing the prototype filter information to generate a surround signal and can be represented as FL_L1 or the like. Rendering information means the filter information necessary for executing rendering and can be represented as HL_L or the like. Interpolated/smoothed rendering information means the filter information resulting from interpolation/smoothing the rendering information and can be represented as HL_L or the like. In the present specification, the above filter informations are referred to. Yet, the present invention is not restricted by the names of the filter informations. In particular, HRTF is taken as an example of the filter information. Yet, the present invention is not limited to the HRTF.
- The
rendering unit 900 receives the decoded downmix signal and the rendering information and then generates a surround signal using the decoded downmix signal and the rendering information. The surround signal may be the signal for providing a surround effect to an audio system capable of generating only a stereo signal. Besides, the present invention can be applied to various systems as well as the audio system capable of generating only the stereo signal. -
FIG. 2 is a structural diagram for a bitstream of an audio signal according to one embodiment of the present invention, in which the bitstream includes an encoded downmix signal and encoded spatial information. - Referring to
FIG. 2 , a 1-frame audio payload includes a downmix signal field and an ancillary data field. Encoded spatial information can be stored in the ancillary data field. For instance, if an audio payload is 48˜128 kbps, spatial information can have a range of 5˜32 kbps. Yet, no limitations are put on the ranges of the audio payload and spatial information. -
FIG. 3 is a detailed block diagram of a spatial information converting unit according to one embodiment of the present invention. - Referring to
FIG. 3 , a spatialinformation converting unit 1000 includes asource mapping unit 1010, a sub-renderinginformation generating unit 1020, an integratingunit 1030, aprocessing unit 1040, and adomain converting unit 1050. - The source mapping unit 101 generates source mapping information corresponding to each source of an audio signal by executing source mapping using spatial information. In this case, the source mapping information means per-source information generated to correspond to each source of an audio signal by using spatial information and the like. The source includes a channel and, in this case, the source mapping information corresponding to each channel is generated. The source mapping information can be represented as a coefficient. And, the source mapping process will be explained in detail later with reference to
FIG. 4 andFIG. 5 . - The sub-rendering
information generating unit 1020 generates sub-rendering information corresponding to each source by using the source mapping information and the filter information. For instance, if therendering unit 900 is the HRTF filter, the sub-renderinginformation generating unit 1020 is able to generate sub-rendering information by using HRTF filter information. - The integrating
unit 1030 generates rendering information by integrating the sub-rendering information to correspond to each source of a downmix signal. The rendering information, which is generated by using the spatial information and the filter information, means the information to generate a surround signal by being applied to the downmix signal. And, the rendering information includes a filter coefficient type. The integration can be omitted to reduce an operation quantity of the rendering process. Subsequently, the rendering information is transferred to theprocessing unit 1042. - The
processing unit 1042 includes aninterpolating unit 1041 and/or asmoothing unit 1042. The rendering information is interpolated by theinterpolating unit 1041 and/or smoothed by thesmoothing unit 1042. - The
domain converting unit 1050 converts a domain of the rendering information to a domain of the downmix signal used by therendering unit 900. And, thedomain converting unit 1050 can be provided to one of various positions including the position shown inFIG. 3 . So, if the rendering information is generated on the same domain of therendering unit 900, it is able to omit thedomain converting unit 1050. The domain-converted rendering information is then transferred to therendering unit 900. - The spatial
information converting unit 1000 can include a filterinformation converting unit 1060. InFIG. 3 , the filterinformation converting unit 1060 is provided within the spatialinformation converting unit 100. Alternatively, the filterinformation converting unit 1060 can be provided outside the spatialinformation converting unit 100. The filterinformation converting unit 1060 is converted to be suitable for generating sub-rendering information or rendering information from random filter information, e.g., HRTF. The converting process of the filter information can include the following steps. - First of all, a step of matching a domain to be applicable is included. If a domain of filter information does not match a domain for executing rendering, the domain matching step is required. For instance, a step of converting time domain HRTF to DFT, QMF or hybrid domain for generating rendering information is necessary.
- Secondly, a coefficient reducing step can be included. In this case, it is easy to save the domain-converted HRTF and apply the domain-converted HRTF to spatial information. For instance, if a prototype filter coefficient has a response of a long tap number (length), a corresponding coefficient has to be stored in a memory corresponding to a response amounting to a corresponding length of total 10 in case of 5.1 channels. This increases a load of the memory and an operational quantity. To prevent this problem, a method of reducing a filter coefficient to be stored while maintaining filter characteristics in the domain converting process can be used. For instance, the HRTF response can be converted to a few parameter value. In this case, a parameter generating process and a parameter value can differ according to an applied domain.
- The downmix signal passes through a domain converting unit 1110 and/or a
decorrelating unit 1200 before being rendered with the rendering information. In case that a domain of the rendering information is different from that of the downmix signal, the domain converting unit 1110 converts the domain of the downmix signal in order to match the two domains together. - The
decorrelating unit 1200 is applied to the domain-converted downmix signal. This may have an operational quantity relatively higher than that of a method of applying a decorrelator to the rendering information. Yet, it is able to prevent distortions from occurring in the process of generating rendering information. Thedecorrelating unit 1200 can include a plurality of decorrelators differing from each other in characteristics if an operational quantity is allowable. If the downmix signal is a stereo signal, thedecorrelating unit 1200 may not be used. InFIG. 3 , in case that a domain-converted mono downmix signal, i.e., a mono downmix signal on a frequency, hybrid, QMF or DFT domain is used in the rendering process, a decorrelator is used on the corresponding domain. And, the present invention includes a decorrelator used on a time domain as well. In this case, a mono downmix signal before thedomain converting unit 1100 is directly inputted to thedecorrelating unit 1200. A first order or higher IIR filter (or FIR filter) is usable as the decorrelator. - Subsequently, the
rendering unit 900 generates a surround signal using the downmix signal, the decorrelated downmix signal, and the rendering information. If the downmix signal is a stereo signal, the decorrelated downmix signal may not be used. Details of the rendering process will be described later with reference toFIGS. 6 to 9 . - The surround signal is converted to a time domain by an inverse
domain converting unit 1300 and then outputted. If so, a user is able to listen to a sound having a multi-channel effect though stereophonic earphones or the like. -
FIG. 4 andFIG. 5 are block diagrams of channel configurations used for source mapping process according to one embodiment of the present invention. A source mapping process is a process for generating source mapping information corresponding to each source of an audio signal by using spatial information. As mentioned in the foregoing description, the source includes a channel and source mapping information can be generated to correspond to the channels shown inFIG. 4 andFIG. 5 . The source mapping information is generated in a type suitable for a rendering process. - For instance, if a downmix signal is a mono signal, it is able to generate source mapping information using spatial information such as CLD1˜CLD5, ICC1˜ICC5, and the like.
- The source mapping information can be represented as such a value as D_L (=DL), D_R (=DR), D_C (=DC), D_LFE (=DLFE), D_Ls (=DLs), D_Rs (=DRs), and the like. In this case, the process for generating the source mapping information is variable according to a tree structure corresponding to spatial information, a range of spatial information to be used, and the like. In the present specification, the downmix signal is a mono signal for example, which does not put limitation of the present invention.
- Right and left channel outputs outputted from the
rendering unit 900 can be expressed as Math Figure 1. -
Lo=L*GL — L′+C*GC — L′+R*GR — L′+Ls*GLs — L′+Rs*GRs — L′ -
Ro=L*GL — R′+C*GC — R′+R*GR — R′+Ls*GLs — R′+Rs*GRs — R′ MathFigure 1 - In this case, the operator ‘*’ indicates a product on a DFT domain and can be replaced by a convolution on a QMF or time domain.
- The present invention includes a method of generating the L, C, R, Ls and Rs by source mapping information using spatial information or by source mapping information using spatial information and filter information. For instance, source mapping information can be generated using CLD of spatial information only or CLD and ICC of spatial information. The method of generating source mapping information using the CLD only is explained as follows.
-
-
- In this case,
-
- and ‘m’ indicates a mono downmix signal.
-
-
- If source mapping information is generated using CLD only, a 3-dimensional effect may be reduced. So, it is able to generate source mapping information using ICC and/or decorrelator. And, a multi-channel signal generated by using a decorrelator output signal dx(m) can be expresses as Math Figure 4.
-
- In this case, ‘A’, ‘B’ and ‘C’ are values that can be represented by using CLD and ICC. ‘d0’ to ‘d3’ indicate decorrelators. And, ‘m’ indicates a mono downmix signal. Yet, this method is unable to generate source mapping information such as D_L, D_R, and the like.
- Hence, the first method of generating the source mapping information using the CLD, ICC and/or decorrelators for the downmix signal regards dx(m) (x=0, 1, 2) as an independent input. In this case, the ‘dx’ is usable for a process for generating sub-rendering filter information according to Math Figure 5.
-
FL — L — M=d — L — M*GL — L′(Mono input→Left output) -
FL — R M=d — L — M*GL — R′(Mono input→Right output) -
FL — L — Dx=d — L — Dx*GL — L′(Dx output→Left output) -
FL — R — Dx=d — L — Dx*GL — R′(Dx output→Right output) MathFigure 5 - And, rendering information can be generated according to Math Figure 6 using a result of Math Figure 5.
-
HM — L=FL — L — M+FR — L — M+FC — L — M+FLS — L — M+FRS — L — M+FLFE — L — M -
HM — R=FL — R — M+FR — R — M+FC — R — M+FLS — R — M+FRS — R — M+FLFE — R — M -
HDx — L=FL — L — Dx+FR — L — Dx+FC — L — Dx+FLS — L — Dx+FRS — L — Dx+FLFE — L — Dx -
HDx — R=FL — R — Dx+FR — R — Dx+FC — R — Dx+FLS — R — Dx+FRS — R — Dx+FLFE — R — Dx MathFigure 6 - Details of the rendering information generating process are explained later. The first method of generating the source mapping information using the CLD, ICC and/or decorrelators handles a dx output value, i.e., ‘dx(m)’ as an independent input, which may increase an operational quantity.
- A second method of generating source mapping information using CLD, ICC and/or decorrelators employs decorrelators applied on a frequency domain. In this case, the source mapping information can be expresses as Math Figure 7.
-
- In this case, by applying decorrelators on a frequency domain, the same source mapping information such as D_L, D_R, and the like before the application of the decorrelators can be generated. So, it can be implemented in a simple manner.
- A third method of generating source mapping information using CLD, ICC and/or decorrelators employs decorrelators having the all-pass characteristic as the decorrelators of the second method. In this case, the all-pass characteristic means that a size is fixed with a phase variation only. And, the present invention can use decorrelators having the all-pass characteristic as the decorrelators of the first method.
- A fourth method of generating source mapping information using CLD, ICC and/or decorrelators carries out decorrelation by using decorrelators for the respective channels (e.g., L, R, C, Ls, Rs, etc.) instead of using ‘d0’ to ‘d3’ of the second method. In this case, the source mapping information can be expressed as Math Figure 8.
-
- In this case, ‘k’ is an energy value of a decorrelated signal determined from CLD and ICC values. And, ‘d_L’, ‘d_R’, ‘d_C’, ‘d_Ls’ and ‘d_Rs’ indicate decorrelators applied to channels, respectively.
- A fifth method of generating source mapping information using CLD, ICC and/or decorrelators maximizes a decorrelation effect by configuring ‘d_L’ and ‘d_R’ symmetric to each other in the fourth method and configuring ‘d_Ls’ and ‘d_Rs’ symmetric to each other in the fourth method. In particular, assuming d_R=f(d_L) and d_Rs=f(d_Ls), it is necessary to design ‘d_L’, ‘d_C’ and ‘d_Ls’ only.
- A sixth method of generating source mapping information using CLD, ICC and/or decorrelators is to configure the ‘d_L’ and ‘d_Ls’ to have a correlation in the fifth method. And, the ‘d_L’ and ‘d_C’ can be configured to have a correlation as well.
- A seventh method of generating source mapping information using CLD, ICC and/or decorrelators is to use the decorrelators in the third method as a serial or nested structure of the all-pas filters. The seventh method utilizes a fact that the all-pass characteristic is maintained even if the all-pass filter is used as the serial or nested structure. In case of using the all-pass filter as the serial or nested structure, it is able to obtain more various kinds of phase responses. Hence, the decorrelation effect can be maximized.
- An eighth method of generating source mapping information using CLD, ICC and/or decorrelators is to use the related art decorrelator and the frequency-domain decorrelator of the second method together. In this case, a multi-channel signal can be expressed as Math Figure 9.
-
- In this case, a filter coefficient generating process uses the same process explained in the first method except that ‘A’ is changed into ‘A+Kd’.
- A ninth method of generating source mapping information using CLD, ICC and/or decorrelators is to generate an additionally decorrelated value by applying a frequency domain decorrelator to an output of the related art decorrelator in case of using the related art decorrelator. Hence, it is able to generate source mapping information with a small operational quantity by overcoming the limitation of the frequency domain decorrelator.
- A tenth method of generating source mapping information using CLD, ICC and/or decorrelators is expressed as Math Figure 10.
-
- In this case, ‘di_(m)’(i=L, R, C, Ls, Rs) is a decorrelator output value applied to a channel-i. And, the output value can be processed on a time domain, a frequency domain, a QMF domain, a hybrid domain, or the like. If the output value is processed on a domain different from a currently processed domain, it can be converted by domain conversion. It is able to use the same ‘d for d_L, d_R, d_C, d_Ls, and d_Rs. In this case, Math Figure 10 can be expressed in a very simple manner.
- If Math Figure 10 is applied to Math Figure 1, Math Figure 1 can be expressed as Math Figure 11.
-
Lo=HM — L*m+HMD — L*d(m) -
Ro=HM — R*R+HMD — R*d(m) MathFigure 11 - In this case, rendering information HM_L is a value resulting from combining spatial information and filter information to generate a surround signal Lo with an input m. And, rendering information HM_R is a value resulting from combining spatial information and filter information to generate a surround signal Ro with an input m. Moreover, ‘d(m)’ is a decorrelator output value generated by transferring a decorrelator output value on an arbitrary domain to a value on a current domain or a decorrelator output value generated by being processed on a current domain. Rendering information HMD_L is a value indicating an extent of the decorrelator output value d(m) that is added to ‘Lo’ in rendering the d(m), and also a value resulting from combining spatial information and filter information together. Rendering information HMD_R is a value indicating an extent of the decorrelator output value d(m) that is added to ‘Ro’ in rendering the d(m).
- Thus, in order to perform a rendering process on a mono downmix signal, the present invention proposes a method of generating a surround signal by rendering the rendering information generated by combining spatial information and filter information (e.g., HRTF filter coefficient) to a downmix signal and a decorrelated downmix signal. The rendering process can be executed regardless of domains. If ‘d(m)’ is expressed as ‘d*m’ (product operator) being executed on a frequency domain, Math Figure 11 can be expressed as Math Figure 12.
-
Lo=HM — L*m+HMD — L*d*m=HMoverall— L*m -
Ro=HM — R*m+HMD — R*d*m=HMoverall— R*m MathFigure 12 - Thus, in case of performing a rendering process on a downmix signal on a frequency domain, it is ale to minimize an operational quantity in a manner of representing a value resulting from combining spatial information, filter information and decorrelators appropriately as a product form.
-
FIG. 6 andFIG. 7 are detailed block diagrams of a rendering unit for a stereo downmix signal according to one embodiment of the present invention. - Referring to
FIG. 6 , therendering unit 900 includes a rendering unit-A 910 and a rendering unit-B 920. - If a downmix signal is a stereo signal, the spatial
information converting unit 1000 generates rendering information for left and right channels of the downmix signal. The rendering unit-A 910 generates a surround signal by rendering the rendering information for the left channel of the downmix signal to the left channel of the downmix signal. And, the rendering unit-B 920 generates a surround signal by rendering the rendering information for the right channel of the downmix signal to the right channel of the downmix signal. The names of the channels are just exemplary, which does not put limitation on the present invention. - The rendering information can include rendering information delivered to a same channel and rendering information delivered to another channel.
- For instance, the spatial
information converting unit 1000 is able to generate rendering information HL_L and HL_R inputted to the rendering unit for the left channel of the downmix signal, in which rendering information HL_L is delivered to a left output corresponding to the same channel and the rendering information HL_R is delivered to a right output corresponding to the another channel. And, the spatialinformation converting unit 1000 is able to generate rendering information HR_R and HR_L inputted to the rendering unit for the right channel of the downmix signal, in which the rendering information HR_R is delivered to a right output corresponding to the same channel and the rendering information HR_L is delivered to a left output corresponding to the another channel. - Referring to
FIG. 7 , therendering unit 900 includes a rendering unit-1 A 911, a rendering unit-2 A 912, a rendering unit-1 B 921, and a rendering unit-2 B 922. - The
rendering unit 900 receives a stereo downmix signal and rendering information from the spatialinformation converting unit 1000. Subsequently, therendering unit 900 generates a surround signal by rendering the rendering information to the stereo downmix signal. - In particular, the rendering unit-
1 A 911 performs rendering by using rendering information HL_L delivered to a same channel among rendering information for a left channel of a downmix signal. The rendering unit-2 A 912 performs rendering by using rendering information HL_R delivered to a another channel among rendering information for a left channel of a downmix signal. The rendering unit-1 B 921 performs rendering by using rendering information HR_R delivered to a same channel among rendering information for a right channel of a downmix signal. And, the rendering unit-2 B 922 performs rendering by using rendering information HR_L delivered to another channel among rendering information for a right channel of a downmix signal. - In the following description, the rendering information delivered to another channel is named ‘cross-rendering information’ The cross-rendering information HL_R or HR_L is applied to a same channel and then added to another channel by an adder. In this case, the cross-rendering information HL_R and/or HR_L can be zero. If the cross-rendering information HL_R and/or HR_L is zero, it means that no contribution is made to the corresponding path.
- An example of the surround signal generating method shown in
FIG. 6 orFIG. 7 is explained as follows. - First of all, if a downmix signal is a stereo signal, the downmix signal defined as ‘x’, source mapping information generated by using spatial information defined as ‘D’, prototype filter information defined as ‘G’, a multi-channel signal defined as ‘p’ and a surround signal defined as ‘y’ can be represented by matrixes shown in Math Figure 13.
-
- In this case, if the above values are on a frequency domain, they can be developed as follows.
- First of all, the multi-channel signal p, as shown in Math Figure 14, can be expressed as a product between the source mapping information D generated by using the spatial information and the downmix signal x.
-
- The surround signal y, as shown in Math Figure 15, can be generated by rendering the prototype filter information G to the multi-channel signal p.
-
y=G·p MathFigure 15 - In this case, if Math Figure 14 is inserted in the p, it can be generated as Math Figure 16.
-
y=GDx MathFigure 16 - In this case, if rendering information H is defined as H=GD, the surround signal y and the downmix signal x can have a relation of Math Figure 17.
-
- Hence, after the rendering information H has been generated by processing the product between the filter information and the source mapping information, the downmix signal x is multiplied by the rendering information H to generate the surround signal y.
- According to the definition of the rendering information H, the rendering information H can be expressed as Math Figure 18.
-
-
FIG. 8 andFIG. 9 are detailed block diagrams of a rendering unit for a mono downmix signal according to one embodiment of the present invention. - Referring to
FIG. 8 , therendering unit 900 includes a rendering unit-A 930 and a rendering unit-B 940. - If a downmix signal is a mono signal, the spatial
information converting unit 1000 generates rendering information HM_L and HM_R, in which the rendering information HM_L is used in rendering the mono signal to a left channel and the rendering information HM_R is used in rendering the mono signal to a right channel. - The rendering unit-
A 930 applies the rendering information HM_L to the mono downmix signal to generate a surround signal of the left channel. The rendering unit-B 940 applies the rendering information HM_R to the mono downmix signal to generate a surround signal of the right channel. - The
rendering unit 900 in the drawing does not use a decorrelator. Yet, if the rendering unit-A 930 and the rendering unit-B 940 performs rendering by using the rendering information Hmoverall_R and Hmoverall_L defined in Math Figure 12, respectively, it is able to obtain the outputs to which the decorrelator is applied, respectively. - Meanwhile, in case of attempting to obtain an output in a stereo signal instead of a surround signal after completion of the rendering performed on a mono downmix signal, the following two methods are possible.
- The first method is that instead of using rendering information for a surround effect, a value used for a stereo output is used. In this case, it is able to obtain a stereo signal by modifying only the rendering information in the structure shown in
FIG. 3 . - The second method is that in a decoding process for generating a multi-channel signal by using a downmix signal and spatial information, it is able to obtain a stereo signal by performing the decoding process to only a corresponding step to obtain a specific channel number.
- Referring to
FIG. 9 , therendering unit 900 corresponds to a case in which a decorrelated signal is represented as one, i.e., Math Figure 11. Therendering unit 900 includes a rendering unit-1 A 931, a rendering unit-2 A 932, a rendering unit-1 B 941, and a rendering unit-2 B 942. Therendering unit 900 is similar to the rendering unit for the stereo downmix signal except that therendering unit 900 includes therendering units - In case of the stereo downmix signal, it can be interpreted that one of two channels is a decorrelated signal. So, without employing additional decorrelators, it is able to perform a rendering process by using the formerly defined four kinds of rendering information HL_L, HL_R and the like. In particular, the rendering unit-
1 A 931 generates a signal to be delivered to a same channel by applying the rendering information HM_L to a mono downmix signal. The rendering unit-2 A 932 generates a signal to be delivered to another channel by applying the rendering information HM_R to the mono downmix signal. The rendering unit-1 B 941 generates a signal to be delivered to a same channel by applying the rendering information HMD_R to a decorrelated signal. And, the rendering unit-2 B 942 generates a signal to be delivered to another channel by applying the rendering information HMD_L to the decorrelated signal. - If a downmix signal is a mono signal, a downmix signal defined as x, source channel information defined as D, prototype filter information defined as G, a multi-channel signal defined as p, and a surround signal defined as y can be represented by matrixes shown in Math Figure 19.
-
- In this case, the relation between the matrixes is similar to that of the case that the downmix signal is the stereo signal. So its details are omitted.
- Meanwhile, the source mapping information described with reference to
FIG. 4 andFIG. 5 and the rendering information generated by using the source mapping information have values differing per frequency band, parameter band, and/or transmitted timeslot. In this case, if a value of the source mapping information and/or the rendering information has a considerably big difference between neighbor bands or between boundary timeslots, distortion may take place in the rendering process. To prevent the distortion, a smoothing process on a frequency and/or time domain is needed. Another smoothing method suitable for the rendering is usable as well as the frequency domain smoothing and/or the time domain smoothing. And, it is able to use a value resulting from multiplying the source mapping information or the rendering information by a specific gain. -
FIG. 10 andFIG. 11 are block diagrams of a smoothing unit and an expanding unit according to one embodiment of the present invention. - A smoothing method according to the present invention, as shown in
FIG. 10 andFIG. 11 , is applicable to rendering information and/or source mapping information. Yet, the smoothing method is applicable to other type information. In the following description, smoothing on a frequency domain is described. Yet, the present invention includes time domain smoothing as well as the frequency domain smoothing. - Referring to
FIG. 10 andFIG. 11 , thesmoothing unit 1042 is capable of performing smoothing on rendering information and/or source mapping information. A detailed example of a position of the smoothing occurrence will be described with reference toFIGS. 18 to 20 later. - The
smoothing unit 1042 can be configured with an expandingunit 1043, in which the rendering information and/or source mapping information can be expanded into a wider range, for example filter band, than that of a parameter band. In particular, the source mapping information can be expanded to a frequency resolution (e.g., filter band) corresponding to filter information to be multiplied by the filter information (e.g., HRTF filter coefficient). The smoothing according to the present invention is executed prior to or together with the expansion. The smoothing used together with the expansion can employ one of the methods shown inFIGS. 12 to 16 . -
FIG. 12 is a graph to explain a first smoothing method according to one embodiment of the present invention. - Referring to
FIG. 12 , a first smoothing method uses a value having the same size as spatial information in each parameter band. In this case, it is able to achieve a smoothing effect by using a suitable smoothing function. -
FIG. 13 is a graph to explain a second smoothing method according to one embodiment of the present invention. - Referring to
FIG. 13 , a second smoothing method is to obtain a smoothing effect by connecting representative positions of parameter band. The representative position is a right center of each of the parameter bands, a central position proportional to a log scale, a bark scale, or the like, a lowest frequency value, or a position previously determined by a different method. -
FIG. 14 is a graph to explain a third smoothing method according to one embodiment of the present invention. - Referring to
FIG. 14 , a third smoothing method is to perform smoothing in a form of a curve or straight line smoothly connecting boundaries of parameters. In this case, the third smoothing method uses a preset boundary smoothing curve or low pass filtering by the first order or higher IIR filter or FIR filter. -
FIG. 15 is a graph to explain a fourth smoothing method according to one embodiment of the present invention. - Referring to
FIG. 15 , a fourth smoothing method is to achieve a smoothing effect by adding a signal such as a random noise to a spatial information contour. In this case, a value differing in channel or band is usable as the random noise. In case of adding a random noise on a frequency domain, it is able to add only a size value while leaving a phase value intact. The fourth smoothing method is able to achieve an inter-channel decorrelation effect as well as a smoothing effect on a frequency domain. -
FIG. 16 is a graph to explain a fifth smoothing method according to one embodiment of the present invention. - Referring to
FIG. 16 , a fifth smoothing method is to use a combination of the second to fourth smoothing methods. For instance, after the representative positions of the respective parameter bands have been connected, the random noise is added and low path filtering is then applied. In doing so, the sequence can be modified. The fifth smoothing method minimizes discontinuous points on a frequency domain and an inter-channel decorrelation effect can be enhanced. - In the first to fifth smoothing methods, a total of powers for spatial information values (e.g., CLD values) on the respective frequency domains per channel should be uniform as a constant. For this, after the smoothing method is performed per channel, power normalization should be performed. For instance, if a downmix signal is a mono signal, level values of the respective channels should meet the relation of Math Figure 20.
-
D — L(pb)+D — R(pb)+D — C(pb)+D — Ls(pb)+D — Rs(pb)+D — Lfe(pb)=C - In this case, ‘pb=0˜ total parameter band number 1’ and ‘C’ is an arbitrary constant.
-
FIG. 17 is a diagram to explain prototype filter information per channel. - Referring to
FIG. 17 , for rendering, a signal having passed through GL_L filter for a left channel source is sent to a left output, whereas a signal having passed through GL_R filter is sent to a right output. - Subsequently, a left final output (e.g., Lo) and a right final output (e.g., Ro) are generated by adding all signals received from the respective channels. In particular, the rendered left/right channel outputs can be expressed as Math Figure 21.
-
Lo=L*GL — L+C*GC — L+R*GR — L+Ls*GLs — L+Rs*GRs — L -
Ro=L*GL — R+C*GC — R+R*GR — R+Ls*GLs — R+Rs*GRs — R MathFigure 21 - In the present invention, the rendered left/right channel outputs can be generated by using the L, R, C, Ls, and Rs generated by decoding the downmix signal into the multi-channel signal using the spatial information. And, the present invention is able to generate the rendered left/right channel outputs using the rendering information without generating the L, R, C, Ls, and Rs, in which the rendering information is generated by using the spatial information and the filter information.
- A process for generating rendering information using spatial information is explained with reference to
FIGS. 18 to 20 as follows. -
FIG. 18 is a block diagram for a first method of generating rendering information in a spatialinformation converting unit 900 according to one embodiment of the present invention. - Referring to
FIG. 18 , as mentioned in the foregoing description, the spatialinformation converting unit 900 includes thesource mapping unit 1010, the sub-renderinginformation generating unit 1020, the integratingunit 1030, theprocessing unit 1040, and thedomain converting unit 1050. The spatialinformation converting unit 900 has the same configuration shown inFIG. 3 . - The sub-rendering
information generating unit 1020 includes at least one or more sub-rendering information generating units (1st sub-rendering information generating unit to Nth sub-rendering information generating unit). - The sub-rendering
information generating unit 1020 generates sub-rendering information by using filter information and source mapping information. - For instance, if a downmix signal is a mono signal, the first sub-rendering information generating unit is able to generate sub-rendering information corresponding to a left channel on a multi-channel. And, the sub-rendering information can be represented as Math Figure 22 using the source mapping information D_L and the converted filter information GL_L′ and GL_R′
-
FL — L=D — L*GL — L′ -
(mono input→filter coefficient to left output channel) -
FL — R=D — L*GL — R′ -
(mono input→filter coefficient to right output channel) MathFigure 22 - In this case, the D_L is a value generated by using the spatial information in the
source mapping unit 1010. Yet, a process for generating the D_L can follow the tree structure. - The second sub-rendering information generating unit is able to generate sub-rendering information FR_L and FR_R corresponding to a right channel on the multi-channel. And, the Nth sub-rendering information generating unit is able to generate sub-rendering information FRs_L and FRs_R corresponding to a right surround channel on the multi-channel.
- If a downmix signal is a stereo signal, the first sub-rendering information generating unit is able to generate sub-rendering information corresponding to the left channel on the multi-channel. And, the sub-rendering information can be represented as Math Figure 23 by using the source mapping information D_L1 and D_L2.
-
FL — L1=D — L1*GL — L′ -
(left input→filter coefficient to left output channel) -
FL — L2=D — L2*GL — L′ -
(right input→filter coefficient to left output channel) -
FL — R1=D — L1*GL — R′ -
(left input→filter coefficient to right output channel) -
FL — R2=D — L2*GL — R′ -
(right input→filter coefficient to right output channel) MathFigure 23 - In Math Figure 23, the FL_R1 is explained for example as follows.
- First of all, in the FL_R1, ‘L’ indicates a position of the multi-channel, ‘R’ indicates an output channel of a surround signal, and ‘1’ indicates a channel of the downmix signal. Namely, the FL_R1 indicates the sub-rendering information used in generating the right output channel of the surround signal from the left channel of the downmix signal.
- Secondly, the D_L1 and the D_L2 are values generated by using the spatial information in the
source mapping unit 1010. - If a downmix signal is a stereo signal, it is able to generate a plurality of sub-rendering informations from at least one sub-rendering information generating unit in the same manner of the case that the downmix signal is the mono signal. The types of the sub-rendering informations generated by a plurality of the sub-rendering information generating units are exemplary, which does not put limitation on the present invention.
- The sub-rendering information generated by the sub-rendering
information generating unit 1020 is transferred to therendering unit 900 via the integratingunit 1030, theprocessing unit 1040, and thedomain converting unit 1050. - The integrating
unit 1030 integrates the sub-rendering informations generated per channel into rendering information (e.g., HL_L, HL_R, HR_L, HR_R) for a rendering process. An integrating process in the integratingunit 1030 is explained for a case of a mono signal and a case of a stereo signal as follows. - First of all, if a downmix signal is a mono signal, rendering information can be expressed as Math Figure 24.
-
HM — L=FL — L+FR — L+FC — L+FLs — L+FRs — L+FLFE — L -
HM — R=FL — R+FR — R+FC — R+FLs — R+FRs — R+FLFE — R MathFigure 24 - Secondly, if a downmix signal is a stereo signal, rendering information can be expressed as Math Figure 25.
-
HL — L=FL — L1+FR — L1+FC — L1+FLs — L1+FRs — L1+FLFE — L1 -
HR — L=FL — L2+FR — L2+FC — L2+FLs — L2+FRs — L2+FLFE — L2 -
HL — R=FL — R1+FR — R1+FC — R1+FLs — R1+FRs — R1+FLFE — R1 -
HR — R=FL — R2+FR — R2+FC — R2+FLs — R2+FRs — R2+FLFE — R2 MathFigure 25 - Subsequently, the
processing unit 1040 includes aninterpolating unit 1041 and/or asmoothing unit 1042 and performs interpolation and/or smoothing for the rendering information. The interpolation and/or smoothing can be executed on a time domain, a frequency domain, or a QMF domain. In the specification, the time domain is taken as an example, which does not put limitation on the present invention. - The interpolation is performed to obtain rendering information non-existing between the rendering informations if the transmitted rendering information has a wide interval on the time domain. For instance, assuming that rendering informations exist in an nth timeslot and an (n+k)th timeslot (k>1), respectively, it is able to perform linear interpolation on a not-transmitted timeslot by using the generated rendering informations (e.g., HL_L, HR_L, HL_R, HR_R).
- The rendering information generated from the interpolation is explained with reference to a case that a downmix signal is a mono signal and a case that the downmix signal is a stereo signal.
- If the downmix signal is the mono signal, the interpolated rendering information can be expressed as Math Figure 26.
-
HM — L(n+j)=HM — L(n)*(1−a)+HM — L(n+k)*a -
HM — R(n+j)=HM — R(n)*(1−a)+HM — R(n+k)*a MathFigure 26 - If the downmix signal is the stereo signal, the interpolated rendering information can be expressed as Math Figure 27.
-
HL — L(n+j)=HL — L(n)*(1−a)+HL — L(n+k)*a -
HR — L(n+j)=HR — L(n)*(1−a)+HR — L(n+k)*a -
HL — R(n+j)=HL — R(n)*(1−a)+HL — R(n+k)*a -
HR — R(n+j)=HR — R(n)*(1−a)+HR — R(n+k)*a MathFigure 27 - In this case, it is 0<j<k. ‘j’ and ‘k’ are integers. And, ‘a’ is a real number corresponding to ‘0<a<1’ to be expressed as Math Figure 28.
-
a=j/k MathFigure 28 - If so, it is able to obtain a value corresponding to the not-transmitted timeslot on a straight line connecting the values in the two timeslots according to Math Figure 27 and Math Figure 28. Details of the interpolation will be explained with reference to
FIG. 22 andFIG. 23 later. In case that a filter coefficient value abruptly varies between two neighboring timeslots on a time domain, thesmoothing unit 1042 executes smoothing to prevent a problem of distortion due to an occurrence of a discontinuous point. The smoothing on the time domain can be carried out using the smoothing method described with reference toFIGS. 12 to 16 . The smoothing can be performed together with expansion. And, the smoothing may differ according to its applied position. If a downmix signal is a mono signal, the time domain smoothing can be represented as Math Figure 29. -
HM — L(n)′=HM — L(n)*b+HM — L(n−1)′*(1−b) -
HM — R(n)′=HM — R(n)*b+HM — R(n−1)′*(1−b) MathFigure 29 - Namely, the smoothing can be executed by the I-pol HR filter type performed in a manner of multiplying the rendering information HM_L(n−1) or HM_R(n−1) smoothed in a previous timeslot n−1 by (1−b), multiplying the rendering information HM_L(n) or HM)R(n) generated in a current timeslot n by b, and adding the two multiplications together. In this case, ‘b’ is a constant for 0<b<1. If ‘b’ gets smaller, a smoothing effect becomes greater. If ‘b’ gets bigger, a smoothing effect becomes smaller. And, the rest of the filters can be applied in the same manner.
- The interpolation and the smoothing can be represented as one expression shown in Math Figure 30 by using Math Figure 29 for the time domain smoothing.
-
- If the interpolation is performed by the
interpolating unit 1041 and/or if the smoothing is performed by thesmoothing unit 1042, rendering information having an energy value different from that of prototype rendering information may be obtained. To prevent this problem, energy normalization may be executed in addition. - Finally, the
domain converting unit 1050 performs domain conversion on the rendering information for a domain for executing the rendering. If the domain for executing the rendering is identical to the domain of rendering information, the domain conversion may not be executed. Thereafter, the domain-converted rendering information is transferred to therendering unit 900. -
FIG. 19 is a block diagram for a second method of generating rendering information in a spatial information converting unit according to one embodiment of the present invention. - The second method is similar to the first method in that a spatial
information converting unit 1000 includes asource mapping unit 1010, a sub-renderinginformation generating unit 1020, an integratingunit 1030, aprocessing unit 1040, and adomain converting unit 1050 and in that the sub-renderinginformation generating unit 1020 includes at least one sub-rendering information generating unit. - Referring to
FIG. 19 , the second method of generating the rendering information differs from the first method in a position of theprocessing unit 1040. So, interpolation and/or smoothing can be performed per channel on sub-rendering informations (e.g., FL_L and FL_R in case of mono signal or FL_L1, FL_L2, FL_R1, FL_R2 in case of stereo signal) generated per channel in the sub-renderinginformation generating unit 1020. - Subsequently, the integrating
unit 1030 integrates the interpolated and/or smoothed sub-rendering informations into rendering information. - The generated rendering information is transferred to the
rendering unit 900 via thedomain converting unit 1050. -
FIG. 20 is a block diagram for a third method of generating rendering filter information in a spatial information converting unit according to one embodiment of the present invention. - The third method is similar to the first or second method in that a spatial
information converting unit 1000 includes asource mapping unit 1010, a sub-renderinginformation generating unit 1020, an integratingunit 1030, aprocessing unit 1040, and adomain converting unit 1050 and in that the sub-renderinginformation generating unit 1020 includes at least one sub-rendering information generating unit. - Referring to
FIG. 20 , the third method of generating the rendering information differs from the first or second method in that theprocessing unit 1040 is located next to thesource mapping unit 1010. So, interpolation and/or smoothing can be performed per channel on source mapping information generated by using spatial information in thesource mapping unit 1010. - Subsequently, the sub-rendering
information generating unit 1020 generates sub-rendering information by using the interpolated and/or smoothed source mapping information and filter information. - The sub-rendering information is integrated into rendering information in the integrating
unit 1030. And, the generated rendering information is transferred to therendering unit 900 via thedomain converting unit 1050. -
FIG. 21 is a diagram to explain a method of generating a surround signal in a rendering unit according to one embodiment of the present invention.FIG. 21 shows a rendering process executed on a DFT domain. Yet, the rendering process can be implemented on a different domain in a similar manner as well.FIG. 21 shows a case that an input signal is a mono downmix signal. Yet,FIG. 21 is applicable to other input channels including a stereo downmix signal and the like in the same manner. - Referring to
FIG. 21 , a mono downmix signal on a time domain preferentially executes windowing having an overlap interval OL in the domain converting unit.FIG. 21 shows a case that 50% overlap is used. Yet, the present invention includes cases of using other overlaps. - A window function for executing the windowing can employ a function having a good frequency selectivity on a DFT domain by being seamlessly connected without discontinuity on a time domain. For instance, a sine square window function can be used as the window function.
- Subsequently, zero padding ZL of a tab length [precisely, (tab length)−1] of a rendering filter using rendering information converted in the domain converting unit is performed on a mono downmix signal having a length OL*2 obtained from the windowing. A domain conversion is then performed into a DFT domain.
FIG. 20 shows that a block-k downmix signal is domain-converted into a DFT domain. - The domain-converted downmix signal is rendered by a rendering filter that uses rendering information. The rendering process can be represented as a product of a downmix signal and rendering information. The rendered downmix signal undergoes IDFT (Inverse Discrete Fourier Transform) in the inverse domain converting unit and is then overlapped with the downmix signal (block k−1 in
FIG. 20 ) previously executed with a delay of a length OL to generate a surround signal. - Interpolation can be performed on each block undergoing the rendering process. The interpolating method is explained as follows.
-
FIG. 22 is a diagram for a first interpolating method according to one embodiment of the present invention. Interpolation according to the present invention can be executed on various positions. For instance, the interpolation can be executed on various positions in the spatial information converting unit shown inFIGS. 18 to 20 or can be executed in the rendering unit. Spatial information, source mapping information, filter information and the like can be used as the values to be interpolated. In the specification, the spatial information is exemplarily used for description. Yet, the present invention is not limited to the spatial information. The interpolation is executed after or together with expansion to a wider band. - Referring to
FIG. 22 , spatial information transferred from an encoding apparatus c an be transferred from a random position instead of being transmitted each timeslot. One spatial frame is able to carry a plurality of spatial information sets (e.g., parameter sets n and n+1 inFIG. 22 ). In case of a low bit rate, one spatial frame is able to carry a single new spatial information set. So, interpolation is carried out for a not-transmitted timeslot using values of a neighboring transmitted spatial information set. An interval between windows for executing rendering does not always match a timeslot. So, an interpolated value at a center of the rendering windows (K−1, K, K+1, K+2, etc.), as shown inFIG. 22 , is found to use. AlthoughFIG. 22 shows that linear interpolation is carried out between timeslots where a spatial information set exists, the present invention is not limited to the interpolating method. For instance, interpolation is not carried out on a timeslot where a spatial information set does not exist. Instead, a previous or preset value can be used. -
FIG. 23 is a diagram for a second interpolating method according to one embodiment of the present invention. - Referring to
FIG. 23 , a second interpolating method according to one embodiment of the present invention has a structure that an interval using a previous value, an interval using a preset default value and the like are combined. For instance, interpolation can be performed by using at least one of a method of maintaining a previous value, a method of using a preset default value, and a method of executing linear interpolation in an interval of one spatial frame. In case that at least two new spatial information sets exist in one window, distortion may take place. In the following description, block switching for preventing the distortion is explained. -
FIG. 24 is a diagram for a block switching method according to one embodiment of the present invention. - Referring to (a) shown in
FIG. 24 , since a window length is greater than a timeslot length, at least two spatial information sets (e.g., parameter sets n and n+1 inFIG. 24 ) can exist in one window interval. In this case, each of the spatial information sets should be applied to a different timeslot. Yet, if one value resulting from interpolating the at least two spatial information sets is applied, distortion may take place. Namely, distortion attributed to time resolution shortage according to a window length can take place. - To solve this problem, a switching method of varying a window size to fit resolution of a timeslot can be used. For instance, a window size, as shown in (b) of
FIG. 24 , can be switched to a shorter-sized window for an interval requesting a high resolution. In this case, at a beginning and an ending portion of switched windows, connecting windows is used to prevent seams from occurring on a time domain of the switched windows. - The window length can be decided by using spatial information in a decoding apparatus instead of being transferred as separate additional information. For instance, a window length can be determined by using an interval of a timeslot for updating spatial information. Namely, if the interval for updating the spatial information is narrow, a window function of short length is used. If the interval for updating the spatial information is wide, a window function of long length is used. In this case, by using a variable length window in rendering, it is advantageous not to use bits for sending window length information separately. Two types of window length are shown in (b) of
FIG. 24 . Yet, windows having various lengths can be used according to transmission frequency and relations of spatial information. The decided window length information is applicable to various steps for generating a surround signal, which is explained in the following description. -
FIG. 25 is a block diagram for a position to which a window length decided by a window length deciding unit is applied according to one embodiment of the present invention. - Referring to
FIG. 25 , a windowlength deciding unit 1400 is able to decide a window length by using spatial information. Information for the decided window length is applicable to asource mapping unit 1010, an integratingunit 1030, aprocessing unit 1040,domain converting units domain converting unit 1300.FIG. 25 shows a case that a stereo downmix signal is used. Yet, the present invention is not limited to the stereo downmix signal only. As mentioned in the foregoing description, even if a window length is shortened, a length of zero padding decided according to a filter tab number is not adjustable. So, a solution for the problem is explained in the following description. -
FIG. 26 is a diagram for filters having various lengths used in processing an audio signal according to one embodiment of the present invention. As mentioned in the foregoing description, if a length of zero padding decided according to a filter tab number is not adjusted, an overlapping amounting to a corresponding length substantially occurs to bring about time resolution shortage. A solution for the problem is to reduce the length of the zero padding by restricting a length of a filter tab. A method of reducing the length of the zero padding can be achieved by truncating a rear portion of a response (e.g., a diffusing interval corresponding to reverberation). In this case, a rendering process may be less accurate than a case of not truncating the rear portion of the filter response. Yet, filter coefficient values on a time domain are very small to mainly affect reverberation. So, a sound quality is not considerably affected by the truncating. - Referring to
FIG. 26 , four kinds of filters are usable. The four kinds of the filters are usable on a DFT domain, which does not put limitation on the present invention. - A filter-N indicates a filter having a long filter length FL and a
length 2*OL of a long zero padding of which filter tab number is not restricted. A filter-N2 indicates a filter having a zeropadding length 2*OL shorter than that of the filter-N1 by restricting a tab number of filter with the same filter length FL. A filter-N3 indicates a filter having a long zeropadding length 2*OL by not restricting a tab number of filter with a filter length FL shorter than that of the filter-N1. And, a filter-N4 indicates a filter having a window length FL shorter than that of the filter-N1 with a short zeropadding length 2*OL by restricting a tab number of filter. - As mentioned in the foregoing description, it is able to solve the problem of time resolution using the above exemplary four kinds of the filters. And, for the rear portion of the filter response, a different filter coefficient is usable for each domain.
-
FIG. 27 is a diagram for a method of processing an audio signal dividedly by using a plurality of subfilters according to one embodiment of the present invention. One filter may be divided into subfilters having filter coefficients differing from each other. After processing the audio signal by using the subfilters, a method of adding results of the processing can be used. In case applying spatial information to a rear portion of a filter response having small energy, i.e., in case of performing rendering by using a filter with a long filter tab, the method provides function for processing dividedly the audio signal by a predetermined length unit. For instance, since the rear portion of the filter response is not considerably varied per HRTF corresponding to each channel, it is able to perform the rendering by extracting a coefficient common to a plurality of windows. In the present specification, a case of execution on a DFT domain is described. Yet, the present invention is not limited to the DFT domain. - Referring to
FIG. 27 , after one filter FL has been divided into a plurality of sub-areas, a plurality of the sub-areas can be processed by a plurality of subfilters (filter-A and filter-B) having filter coefficients differing from each other. - Subsequently, an output processed by the filter-A and an output processed by the filter-B are combined together. For instance, IDFT (Inverse Discrete Fourier Transform) is performed on each of the output processed by the filter-A and the output processed by the filter-B to generate a time domain signal. And, the generated signals are added together. In this case, a position, to which the output processed by the filter-B is added, is time-delayed by FL more than a position of the output processed by the filter-A. In this way, the signal processed by a plurality of the subfilters brings the same effect of the case that the signal is processed by a single filter.
- And, the present invention includes a method of rendering the output processed by the filter-B to a downmix signal directly. In this case, it is able to render the output to the downmix signal by using coefficients extracting from spatial information, the spatial information in part or without using the spatial information.
- The method is characterized in that a filter having a long tab number can be applied dividedly and that a rear portion of the filter having small energy is applicable without conversion using spatial information. In this case, if conversion using spatial information is not applied, a different filter is not applied to each processed window. So, it is unnecessary to apply the same scheme as the block switching.
FIG. 26 shows that the filter is divided into two areas. Yet, the present invention is able to divide the filter into a plurality of areas. -
FIG. 28 is a block diagram for a method of rendering partition rendering information generated by a plurality of subfilters to a mono downmix signal according to one embodiment of the present invention.FIG. 28 relates to one rendering coefficient. The method can be executed per rendering coefficient. - Referring to
FIG. 28 , the filter-A information ofFIG. 27 corresponds to first partition rendering information HM_L_A and the filter-B information ofFIG. 27 corresponds to second partition rendering information HM_L_B.FIG. 28 shows an embodiment of partition into two subfilters. Yet, the present invention is not limited to the two subfilters. The two subfilters can be obtained via asplitting unit 1500 using the rendering information HM_L generated in the spatialinformation generating unit 1000. Alternatively, the two subfilters can be obtained using prototype HRTF information or information decided according to a user's selection. The information decided according to a user's selection may include spatial information selected according to a user's taste for example. In this case, HM_L_A is the rendering information based on the received spatial information and, HM_L_B may be the rendering information for providing a 3-dimensional effect commonly applied to signals. - As mentioned in the foregoing description, the processing with a plurality of the subfilters is applicable to a time domain and a QMF domain as well as the DFT domain. In particular, the coefficient values split by the filter-A and the filter-B are applied to the downmix signal by time or QMF domain rendering and are then added to generate a final signal.
- The
rendering unit 900 includes a firstpartition rendering unit 950 and a secondpartition rendering unit 960. The firstpartition rendering unit 950 performs a rendering process using HM_L_A, whereas the secondpartition rendering unit 960 performs a rendering process using HM_L_B. - If the filter-A and the filter-B, as shown in
FIG. 27 , are splits of a same filter according to time, it is able to consider a proper delay to correspond to the time interval.FIG. 28 shows an example of a mono downmix signal. In case of using mono downmix signal and decorrelator, a portion corresponding to the filter-B is applied not to the decorrelator but to the mono downmix signal directly. -
FIG. 29 is a block diagram for a method of rendering partition rendering information generated using a plurality of subfilters to a stereo downmix signal according to one embodiment of the present invention. - A partition rendering process shown in
FIG. 29 is similar to that ofFIG. 28 in that two subfilters are obtained in asplitter 1500 by using rendering information generated by the spatialinformation converting unit 1000, prototype HRTF filter information or user decision information. The difference fromFIG. 28 lies in that a partition rendering process corresponding to the filter-B is commonly applied to LUR signals. - In particular, the
splitter 1500 generates first partition rendering information corresponding to filter-A information, second partition rendering information, and third partition rendering information corresponding to filter-B information. In this case, the third partition rendering information can be generated by using filter information or spatial information commonly applicable to the LIR signals. - Referring to
FIG. 29 , arendering unit 900 includes a firstpartition rendering unit 970, a secondpartition rendering unit 980, and a thirdpartition rendering unit 990. - The third partition rendering information generates is applied to a sum signal of the L/R signals in the third
partition rendering unit 990 to generate one output signal. The output signal is added to the L/R output signals, which are independently rendered by a filter-A1 and a filter-A2 in the first and secondpartition rendering units partition rendering unit 990 can be added after an appropriate delay. InFIG. 29 , an expression of cross rendering information applied to another channel from L/R inputs is omitted for convenience of explanation. -
FIG. 30 is a block diagram for a first domain converting method of a downmix signal according to one embodiment of the present invention. The rendering process executed on the DFT domain has been described so far. As mentioned in the foregoing description, the rendering process is executable on other domains as well as the DFT domain. Yet,FIG. 30 shows the rendering process executed on the DFT domain. Adomain converting unit 1100 includes a QMF filter and a DFT filter. An inversedomain converting unit 1300 includes an IDFT filter and an IQMF filter.FIG. 30 relates to a mono downmix signal, which does not put limitation on the present invention. - Referring to
FIG. 30 , a time domain downmix signal of p samples passes through a QMF filter to generate P sub-band samples. W samples are recollected per band. After windowing is performed on the recollected samples, zero padding is performed. M-point DFT (FFT) is then executed. In this case, the DFT enables a processing by the aforesaid type windowing. A value connecting the M/2 frequency domain values per band obtained by the M-point DFT to P bands can be regarded as an approximate value of a frequency spectrum obtained by M/2*P-point DFT. So, a filter coefficient represented on a M/2*P-point DFT domain is multiplied by the frequency spectrum to bring the same effect of the rendering process on the DFT domain. - In this case, the signal having passed through the QMF filter has leakage, e.g., aliasing between neighboring bands. In particular, a value corresponding to a neighbor band smears in a current band and a portion of a value existing in the current band is shifted to the neighbor band. In this case, if QMF integration is executed, an original signal can be recovered due to QMF characteristics. Yet, if a filtering process is performed on the signal of the corresponding band as the case in the present invention, the signal is distorted by the leakage. To minimize this problem, a process for recovering an original signal can be added in a manner of having a signal pass through a leakage minimizing butterfly B prior to performing DFT per band after QMF in the
domain converting unit 100 and performing a reversing process V after IDFT in the inversedomain converting unit 1300. - Meanwhile, to match the generating process of the rendering information generated in the spatial
information converting unit 1000 with the generating process of the downmix signal, DFT can be performed on a QMF pass signal for prototype filter information instead of executing M/2*P-point DFT in the beginning. In this case, delay and data spreading due to QMF filter may exist. -
FIG. 31 is a block diagram for a second domain converting method of a downmix signal according to one embodiment of the present invention.FIG. 31 shows a rendering process performed on a QMF domain. - Referring to
FIG. 31 , adomain converting unit 1100 includes a QMF domain converting unit and an inversedomain converting unit 1300 includes an IQMF domain converting unit. A configuration shown inFIG. 31 is equal to that of the case of using DFT only except that the domain converting unit is a QMF filter. In the following description, the QMF is referred to as including a QMF and a hybrid QMF having the same bandwidth. The difference from the case of using DFT only lies in that the generation of the rendering information is performed on the QMF domain and that the rendering process is represented as a convolution instead of the product on the DFT domain, since the rendering process performed by a renderer-M 3012 is executed on the QMF domain. - Assuming that the QMF filter is provided with B bands, a filter coefficient can be represented as a set of filter coefficients having different features (coefficients) for the B bands. Occasionally, if a filter tab number becomes a first order (i.e., multiplied by a constant), a rendering process on a DFT domain having B frequency spectrums and an operational process are matched. Math Figure 31 represents a rendering process executed in one QMF band (b) for one path for performing the rendering process using rendering information HM_L.
-
- In this case, k indicates a time order in QMF band, i.e., a timeslot unit. The rendering process executed on the QMF domain is advantageous in that, if spatial information transmitted is a value applicable to the QMF domain, application of corresponding data is most facilitated and that distortion in the course of application can be minimized. Yet, in case of QMF domain conversion in the prototype filter information (e.g., prototype filter coefficient) converting process, a considerable operational quantity is required for a process of applying the converted value. In this case, the operational quantity can be minimized by the method of parameterizing the HRTF coefficient in the filter information converting process.
- Accordingly, the signal processing method and apparatus of the present invention uses spatial information provided by an encoder to generate surround signals by using HRTF filter information or filter information according to a user in a decoding apparatus in capable of generating multi-channels. And, the present invention is usefully applicable to various kinds of decoders capable of reproducing stereo signals only.
- While the present invention has been described and illustrated herein with reference to the preferred embodiments thereof, it will be apparent to those skilled in the art that various modifications and variations can be made therein without departing from the spirit and scope of the invention. Thus, it is intended that the present invention covers the modifications and variations of this invention that come within the scope of the appended claims and their equivalents.
Claims (19)
1. A method of processing a signal, the method comprising of:
generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources;
generating at least one rendering information by using the source mapping information and filter information having a surround effect; and
performing smoothing by using neighbor rendering information of the at least one rendering information.
2. The method of claim 1 , wherein the smoothing is performed on a time or frequency domain.
3. The method of claim 2 , wherein the smoothing is performed by adding a random noise to the rendering information.
4. The method of claim 3 , wherein a different value of the random noise is added to the rendering information per band in which the smoothing is performed.
5. The method of claim 1 , wherein the smoothing is performed by interpolating the rendering information at a representative position of a band in which the smoothing is performed.
6. The method of claim 5 , wherein the representative position is a center position, a lowest position, or a central position proportional to a log scale/bark scale of the band for performing the smoothing.
7. The method of claim 1 , wherein the smoothing is performed in a curve shape connecting the rendering information.
8. The method of claim 7 , wherein the smoothing is performed by using a pre-determined smoothing curve or a first order or higher low pass filter.
9. The method of claim 1 , wherein the smoothing is performed by interpolating the rendering information at a representative position of a band in which the smoothing is performed, adding a random noise to the rendering information, and then executing low pass filtering.
10. The method of claim 1 , further comprising expanding the rendering information from a first frequency band in which the rendering information is generated to a second frequency band.
11. The method of claim 10 , wherein the rendering information is expanded by using a same value of the rendering information in the first frequency band.
12. The method of claim 1 , wherein a filter used for the smoothing comprises a first order IIR filter.
13. The method of claim 1 , further comprising generating a surround signal by applying the smoothed rendering information to a downmix signal generated by downmixing the multi-sources.
14. The method of claim 13 , wherein the filter information includes HRTF filter information or a value decided according to a user's selection.
15. The method of claim 14 , wherein the filter information is domain-converted into a domain in the surround signal is generated.
16. The method of claim 15 , wherein the filter information is generated by converting the HRTF filter information into a parameter.
17. An apparatus for processing signals, the apparatus comprising:
a source mapping unit generating source mapping information corresponding to each source of multi-sources by using spatial information indicating features between the multi-sources;
an integrating unit generating at least one rendering information by using the source mapping information and filter information having a surround effect; and
a smoothing unit performing smoothing by using neighbor rendering information of the at least one rendering information.
18. The apparatus of claim 17 , further comprising a rendering unit generating a surround signal by applying the smoothed rendering information to a downmix signal generated by downmixing the multi-sources.
19. The apparatus of claim 18 , further comprising a filter information converting unit converting a domain of the filter information including HRTF filter information or a value decided according to a user's selection to a domain in which the surround signal is generated.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/161,560 US20090028344A1 (en) | 2006-01-19 | 2007-01-19 | Method and Apparatus for Processing a Media Signal |
Applications Claiming Priority (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US75998006P | 2006-01-19 | 2006-01-19 | |
US77672406P | 2006-02-27 | 2006-02-27 | |
US77944206P | 2006-03-07 | 2006-03-07 | |
US77944106P | 2006-03-07 | 2006-03-07 | |
US77941706P | 2006-03-07 | 2006-03-07 | |
US78717206P | 2006-03-30 | 2006-03-30 | |
US78751606P | 2006-03-31 | 2006-03-31 | |
US12/161,560 US20090028344A1 (en) | 2006-01-19 | 2007-01-19 | Method and Apparatus for Processing a Media Signal |
PCT/KR2007/000340 WO2007083952A1 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
Publications (1)
Publication Number | Publication Date |
---|---|
US20090028344A1 true US20090028344A1 (en) | 2009-01-29 |
Family
ID=38287846
Family Applications (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/161,329 Active 2029-09-11 US8521313B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,560 Abandoned US20090028344A1 (en) | 2006-01-19 | 2007-01-19 | Method and Apparatus for Processing a Media Signal |
US12/161,558 Active 2030-11-11 US8411869B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,337 Active 2029-11-15 US8351611B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,334 Active 2029-10-24 US8208641B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,563 Active 2030-05-06 US8488819B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/161,329 Active 2029-09-11 US8521313B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
Family Applications After (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/161,558 Active 2030-11-11 US8411869B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,337 Active 2029-11-15 US8351611B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,334 Active 2029-10-24 US8208641B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
US12/161,563 Active 2030-05-06 US8488819B2 (en) | 2006-01-19 | 2007-01-19 | Method and apparatus for processing a media signal |
Country Status (11)
Country | Link |
---|---|
US (6) | US8521313B2 (en) |
EP (6) | EP1979898B1 (en) |
JP (6) | JP4814343B2 (en) |
KR (8) | KR100953640B1 (en) |
AU (1) | AU2007206195B2 (en) |
BR (1) | BRPI0707136A2 (en) |
CA (1) | CA2636494C (en) |
ES (3) | ES2446245T3 (en) |
HK (1) | HK1127433A1 (en) |
TW (7) | TWI329461B (en) |
WO (6) | WO2007083959A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100310081A1 (en) * | 2009-06-08 | 2010-12-09 | Mstar Semiconductor, Inc. | Multi-channel Audio Signal Decoding Method and Device |
US20120092566A1 (en) * | 2010-10-19 | 2012-04-19 | Samsung Electronics Co., Ltd. | Image processing apparatus, sound processing method used for image processing apparatus, and sound processing apparatus |
US8515771B2 (en) | 2009-09-01 | 2013-08-20 | Panasonic Corporation | Identifying an encoding format of an encoded voice signal |
US9060236B2 (en) * | 2009-10-20 | 2015-06-16 | Dolby International Ab | Apparatus for providing an upmix signal representation on the basis of a downmix signal representation, apparatus for providing a bitstream representing a multi-channel audio signal, methods, computer program and bitstream using a distortion control signaling |
US9093080B2 (en) | 2010-06-09 | 2015-07-28 | Panasonic Intellectual Property Corporation Of America | Bandwidth extension method, bandwidth extension apparatus, program, integrated circuit, and audio decoding apparatus |
US10142763B2 (en) | 2013-11-27 | 2018-11-27 | Dolby Laboratories Licensing Corporation | Audio signal processing |
Families Citing this family (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
SE0400998D0 (en) | 2004-04-16 | 2004-04-16 | Cooding Technologies Sweden Ab | Method for representing multi-channel audio signals |
WO2007083959A1 (en) * | 2006-01-19 | 2007-07-26 | Lg Electronics Inc. | Method and apparatus for processing a media signal |
GB2452021B (en) * | 2007-07-19 | 2012-03-14 | Vodafone Plc | identifying callers in telecommunication networks |
KR101464977B1 (en) * | 2007-10-01 | 2014-11-25 | 삼성전자주식회사 | Method of managing a memory and Method and apparatus of decoding multi channel data |
ES2391801T3 (en) * | 2008-01-01 | 2012-11-30 | Lg Electronics Inc. | Procedure and apparatus for processing an audio signal |
EP2232486B1 (en) | 2008-01-01 | 2013-07-17 | LG Electronics Inc. | A method and an apparatus for processing an audio signal |
KR101061129B1 (en) * | 2008-04-24 | 2011-08-31 | 엘지전자 주식회사 | Method of processing audio signal and apparatus thereof |
US20110112843A1 (en) * | 2008-07-11 | 2011-05-12 | Nec Corporation | Signal analyzing device, signal control device, and method and program therefor |
EP2175670A1 (en) * | 2008-10-07 | 2010-04-14 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Binaural rendering of a multi-channel audio signal |
MX2011011399A (en) * | 2008-10-17 | 2012-06-27 | Univ Friedrich Alexander Er | Audio coding using downmix. |
EP2214162A1 (en) * | 2009-01-28 | 2010-08-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Upmixer, method and computer program for upmixing a downmix audio signal |
US8396576B2 (en) * | 2009-08-14 | 2013-03-12 | Dts Llc | System for adaptively streaming audio objects |
KR101692394B1 (en) * | 2009-08-27 | 2017-01-04 | 삼성전자주식회사 | Method and apparatus for encoding/decoding stereo audio |
TWI557723B (en) * | 2010-02-18 | 2016-11-11 | 杜比實驗室特許公司 | Decoding method and system |
CA3025108C (en) | 2010-07-02 | 2020-10-27 | Dolby International Ab | Audio decoding with selective post filtering |
US20120035940A1 (en) * | 2010-08-06 | 2012-02-09 | Samsung Electronics Co., Ltd. | Audio signal processing method, encoding apparatus therefor, and decoding apparatus therefor |
US8948403B2 (en) * | 2010-08-06 | 2015-02-03 | Samsung Electronics Co., Ltd. | Method of processing signal, encoding apparatus thereof, decoding apparatus thereof, and signal processing system |
US8908874B2 (en) * | 2010-09-08 | 2014-12-09 | Dts, Inc. | Spatial audio encoding and reproduction |
PT2429208T (en) * | 2010-09-09 | 2021-02-04 | Mk Systems Usa Inc Usa Inc | Video bit-rate control |
WO2012122397A1 (en) | 2011-03-09 | 2012-09-13 | Srs Labs, Inc. | System for dynamically creating and rendering audio objects |
KR101842257B1 (en) * | 2011-09-14 | 2018-05-15 | 삼성전자주식회사 | Method for signal processing, encoding apparatus thereof, and decoding apparatus thereof |
US9317458B2 (en) * | 2012-04-16 | 2016-04-19 | Harman International Industries, Incorporated | System for converting a signal |
EP2717265A1 (en) | 2012-10-05 | 2014-04-09 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Encoder, decoder and methods for backward compatible dynamic adaption of time/frequency resolution in spatial-audio-object-coding |
TWI618051B (en) | 2013-02-14 | 2018-03-11 | 杜比實驗室特許公司 | Audio signal processing method and apparatus for audio signal enhancement using estimated spatial parameters |
EP2956935B1 (en) | 2013-02-14 | 2017-01-04 | Dolby Laboratories Licensing Corporation | Controlling the inter-channel coherence of upmixed audio signals |
TWI618050B (en) | 2013-02-14 | 2018-03-11 | 杜比實驗室特許公司 | Method and apparatus for signal decorrelation in an audio processing system |
WO2014126688A1 (en) | 2013-02-14 | 2014-08-21 | Dolby Laboratories Licensing Corporation | Methods for audio signal transient detection and decorrelation control |
EP2981955B1 (en) | 2013-04-05 | 2023-06-07 | Dts Llc | Layered audio coding and transmission |
WO2015006112A1 (en) * | 2013-07-08 | 2015-01-15 | Dolby Laboratories Licensing Corporation | Processing of time-varying metadata for lossless resampling |
EP2830332A3 (en) | 2013-07-22 | 2015-03-11 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method, signal processing unit, and computer program for mapping a plurality of input channels of an input channel configuration to output channels of an output channel configuration |
EP2830333A1 (en) | 2013-07-22 | 2015-01-28 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Multi-channel decorrelator, multi-channel audio decoder, multi-channel audio encoder, methods and computer program using a premix of decorrelator input signals |
EP2830051A3 (en) | 2013-07-22 | 2015-03-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio encoder, audio decoder, methods and computer program using jointly encoded residual signals |
EP3022949B1 (en) | 2013-07-22 | 2017-10-18 | Fraunhofer Gesellschaft zur Förderung der angewandten Forschung E.V. | Multi-channel audio decoder, multi-channel audio encoder, methods, computer program and encoded audio representation using a decorrelation of rendered audio signals |
KR20230011480A (en) * | 2013-10-21 | 2023-01-20 | 돌비 인터네셔널 에이비 | Parametric reconstruction of audio signals |
EP2866227A1 (en) * | 2013-10-22 | 2015-04-29 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method for decoding and encoding a downmix matrix, method for presenting audio content, encoder and decoder for a downmix matrix, audio encoder and audio decoder |
US10754925B2 (en) | 2014-06-04 | 2020-08-25 | Nuance Communications, Inc. | NLU training with user corrections to engine annotations |
US10373711B2 (en) | 2014-06-04 | 2019-08-06 | Nuance Communications, Inc. | Medical coding system with CDI clarification request notification |
EP2980789A1 (en) * | 2014-07-30 | 2016-02-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for enhancing an audio signal, sound enhancing system |
US10490197B2 (en) * | 2015-06-17 | 2019-11-26 | Samsung Electronics Co., Ltd. | Method and device for processing internal channels for low complexity format conversion |
KR102627374B1 (en) * | 2015-06-17 | 2024-01-19 | 삼성전자주식회사 | Internal channel processing method and device for low-computation format conversion |
US10366687B2 (en) * | 2015-12-10 | 2019-07-30 | Nuance Communications, Inc. | System and methods for adapting neural network acoustic models |
EP3516560A1 (en) | 2016-09-20 | 2019-07-31 | Nuance Communications, Inc. | Method and system for sequencing medical billing codes |
US11133091B2 (en) | 2017-07-21 | 2021-09-28 | Nuance Communications, Inc. | Automated analysis system and method |
US11024424B2 (en) | 2017-10-27 | 2021-06-01 | Nuance Communications, Inc. | Computer assisted coding systems and methods |
CN109859766B (en) | 2017-11-30 | 2021-08-20 | 华为技术有限公司 | Audio coding and decoding method and related product |
WO2019241760A1 (en) * | 2018-06-14 | 2019-12-19 | Magic Leap, Inc. | Methods and systems for audio signal filtering |
Citations (97)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5166685A (en) * | 1990-09-04 | 1992-11-24 | Motorola, Inc. | Automatic selection of external multiplexer channels by an A/D converter integrated circuit |
US5524054A (en) * | 1993-06-22 | 1996-06-04 | Deutsche Thomson-Brandt Gmbh | Method for generating a multi-channel audio decoder matrix |
US5561736A (en) * | 1993-06-04 | 1996-10-01 | International Business Machines Corporation | Three dimensional speech synthesis |
US5579396A (en) * | 1993-07-30 | 1996-11-26 | Victor Company Of Japan, Ltd. | Surround signal processing apparatus |
US5632005A (en) * | 1991-01-08 | 1997-05-20 | Ray Milton Dolby | Encoder/decoder for multidimensional sound fields |
US5668924A (en) * | 1995-01-18 | 1997-09-16 | Olympus Optical Co. Ltd. | Digital sound recording and reproduction device using a coding technique to compress data for reduction of memory requirements |
US5703584A (en) * | 1994-08-22 | 1997-12-30 | Adaptec, Inc. | Analog data acquisition system |
US5886988A (en) * | 1996-10-23 | 1999-03-23 | Arraycomm, Inc. | Channel assignment and call admission control for spatial division multiple access communication systems |
US5890125A (en) * | 1997-07-16 | 1999-03-30 | Dolby Laboratories Licensing Corporation | Method and apparatus for encoding and decoding multiple audio channels at low bit rates using adaptive selection of encoding method |
US6072877A (en) * | 1994-09-09 | 2000-06-06 | Aureal Semiconductor, Inc. | Three-dimensional virtual audio display employing reduced complexity imaging filters |
US6118875A (en) * | 1994-02-25 | 2000-09-12 | Moeller; Henrik | Binaural synthesis, head-related transfer functions, and uses thereof |
US6122619A (en) * | 1998-06-17 | 2000-09-19 | Lsi Logic Corporation | Audio decoder with programmable downmixing of MPEG/AC-3 and method therefor |
US6226616B1 (en) * | 1999-06-21 | 2001-05-01 | Digital Theater Systems, Inc. | Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility |
US20010031062A1 (en) * | 2000-02-02 | 2001-10-18 | Kenichi Terai | Headphone system |
US6307941B1 (en) * | 1997-07-15 | 2001-10-23 | Desper Products, Inc. | System and method for localization of virtual sound |
US6466913B1 (en) * | 1998-07-01 | 2002-10-15 | Ricoh Company, Ltd. | Method of determining a sound localization filter and a sound localization control system incorporating the filter |
US6574339B1 (en) * | 1998-10-20 | 2003-06-03 | Samsung Electronics Co., Ltd. | Three-dimensional sound reproducing apparatus for multiple listeners and method thereof |
US6611212B1 (en) * | 1999-04-07 | 2003-08-26 | Dolby Laboratories Licensing Corp. | Matrix improvements to lossless encoding and decoding |
US6633648B1 (en) * | 1999-11-12 | 2003-10-14 | Jerald L. Bauck | Loudspeaker array for enlarged sweet spot |
US20030236583A1 (en) * | 2002-06-24 | 2003-12-25 | Frank Baumgarte | Hybrid multi-channel/cue coding/decoding of audio signals |
US20040032960A1 (en) * | 2002-05-03 | 2004-02-19 | Griesinger David H. | Multichannel downmixing device |
US20040049379A1 (en) * | 2002-09-04 | 2004-03-11 | Microsoft Corporation | Multi-channel audio encoding and decoding |
US6711266B1 (en) * | 1997-02-07 | 2004-03-23 | Bose Corporation | Surround sound channel encoding and decoding |
US6721425B1 (en) * | 1997-02-07 | 2004-04-13 | Bose Corporation | Sound signal mixing |
US20040071445A1 (en) * | 1999-12-23 | 2004-04-15 | Tarnoff Harry L. | Method and apparatus for synchronization of ancillary information in film conversion |
US20040111171A1 (en) * | 2002-10-28 | 2004-06-10 | Dae-Young Jang | Object-based three-dimensional audio system and method of controlling the same |
US20040138874A1 (en) * | 2003-01-09 | 2004-07-15 | Samu Kaajas | Audio signal processing |
US6795556B1 (en) * | 1999-05-29 | 2004-09-21 | Creative Technology, Ltd. | Method of modifying one or more original head related transfer functions |
US20040196770A1 (en) * | 2002-05-07 | 2004-10-07 | Keisuke Touyama | Coding method, coding device, decoding method, and decoding device |
US20050074127A1 (en) * | 2003-10-02 | 2005-04-07 | Jurgen Herre | Compatible multi-channel coding/decoding |
US20050089181A1 (en) * | 2003-10-27 | 2005-04-28 | Polk Matthew S.Jr. | Multi-channel audio surround sound from front located loudspeakers |
US20050117762A1 (en) * | 2003-11-04 | 2005-06-02 | Atsuhiro Sakurai | Binaural sound localization using a formant-type cascade of resonators and anti-resonators |
US20050135643A1 (en) * | 2003-12-17 | 2005-06-23 | Joon-Hyun Lee | Apparatus and method of reproducing virtual sound |
US20050157883A1 (en) * | 2004-01-20 | 2005-07-21 | Jurgen Herre | Apparatus and method for constructing a multi-channel output signal or for generating a downmix signal |
US20050180579A1 (en) * | 2004-02-12 | 2005-08-18 | Frank Baumgarte | Late reverberation-based synthesis of auditory scenes |
US20050179701A1 (en) * | 2004-02-13 | 2005-08-18 | Jahnke Steven R. | Dynamic sound source and listener position based audio rendering |
US20050273324A1 (en) * | 2004-06-08 | 2005-12-08 | Expamedia, Inc. | System for providing audio data and providing method thereof |
US20050273322A1 (en) * | 2004-06-04 | 2005-12-08 | Hyuck-Jae Lee | Audio signal encoding and decoding apparatus |
US20060004583A1 (en) * | 2004-06-30 | 2006-01-05 | Juergen Herre | Multi-channel synthesizer and method for generating a multi-channel output signal |
US20060002572A1 (en) * | 2004-07-01 | 2006-01-05 | Smithers Michael J | Method for correcting metadata affecting the playback loudness and dynamic range of audio information |
US20060009225A1 (en) * | 2004-07-09 | 2006-01-12 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | Apparatus and method for generating a multi-channel output signal |
US20060008094A1 (en) * | 2004-07-06 | 2006-01-12 | Jui-Jung Huang | Wireless multi-channel audio system |
US20060072764A1 (en) * | 2002-11-20 | 2006-04-06 | Koninklijke Philips Electronics N.V. | Audio based data representation apparatus and method |
US20060115100A1 (en) * | 2004-11-30 | 2006-06-01 | Christof Faller | Parametric coding of spatial audio with cues based on transmitted channels |
US20060126851A1 (en) * | 1999-10-04 | 2006-06-15 | Yuen Thomas C | Acoustic correction apparatus |
US20060133618A1 (en) * | 2004-11-02 | 2006-06-22 | Lars Villemoes | Stereo compatible multi-channel audio coding |
US7085393B1 (en) * | 1998-11-13 | 2006-08-01 | Agere Systems Inc. | Method and apparatus for regularizing measured HRTF for smooth 3D digital audio |
US20060195981A1 (en) * | 2005-03-02 | 2006-09-07 | Hydro-Industries Tynat Ltd. | Freestanding combination sink and hose reel workstation |
US20060198527A1 (en) * | 2005-03-03 | 2006-09-07 | Ingyu Chun | Method and apparatus to generate stereo sound for two-channel headphones |
US20060233380A1 (en) * | 2005-04-15 | 2006-10-19 | FRAUNHOFER- GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG e.V. | Multi-channel hierarchical audio coding with compact side information |
US20060233379A1 (en) * | 2005-04-15 | 2006-10-19 | Coding Technologies, AB | Adaptive residual audio coding |
US20060239473A1 (en) * | 2005-04-15 | 2006-10-26 | Coding Technologies Ab | Envelope shaping of decorrelated signals |
US7177431B2 (en) * | 1999-07-09 | 2007-02-13 | Creative Technology, Ltd. | Dynamic decorrelator for audio signals |
US7180964B2 (en) * | 2002-06-28 | 2007-02-20 | Advanced Micro Devices, Inc. | Constellation manipulation for frequency/phase error correction |
US20070133831A1 (en) * | 2005-09-22 | 2007-06-14 | Samsung Electronics Co., Ltd. | Apparatus and method of reproducing virtual sound of two channels |
WO2007068243A1 (en) * | 2005-12-16 | 2007-06-21 | Widex A/S | Method and system for surveillance of a wireless connection in a hearing aid fitting system |
US20070160218A1 (en) * | 2006-01-09 | 2007-07-12 | Nokia Corporation | Decoding of binaural audio signals |
US20070162278A1 (en) * | 2004-02-25 | 2007-07-12 | Matsushita Electric Industrial Co., Ltd. | Audio encoder and audio decoder |
US20070165886A1 (en) * | 2003-11-17 | 2007-07-19 | Richard Topliss | Louderspeaker |
US20070172071A1 (en) * | 2006-01-20 | 2007-07-26 | Microsoft Corporation | Complex transforms for multi-channel audio |
US20070183603A1 (en) * | 2000-01-17 | 2007-08-09 | Vast Audio Pty Ltd | Generation of customised three dimensional sound effects for individuals |
US7260540B2 (en) * | 2001-11-14 | 2007-08-21 | Matsushita Electric Industrial Co., Ltd. | Encoding device, decoding device, and system thereof utilizing band expansion information |
US20070203697A1 (en) * | 2005-08-30 | 2007-08-30 | Hee Suk Pang | Time slot position coding of multiple frame types |
US20070223708A1 (en) * | 2006-03-24 | 2007-09-27 | Lars Villemoes | Generation of spatial downmixes from parametric representations of multi channel signals |
US20070223709A1 (en) * | 2006-03-06 | 2007-09-27 | Samsung Electronics Co., Ltd. | Method, medium, and system generating a stereo signal |
US20070233296A1 (en) * | 2006-01-11 | 2007-10-04 | Samsung Electronics Co., Ltd. | Method, medium, and apparatus with scalable channel decoding |
US20070258607A1 (en) * | 2004-04-16 | 2007-11-08 | Heiko Purnhagen | Method for representing multi-channel audio signals |
US7302068B2 (en) * | 2001-06-21 | 2007-11-27 | 1 . . .Limited | Loudspeaker |
US20070280485A1 (en) * | 2006-06-02 | 2007-12-06 | Lars Villemoes | Binaural multi-channel decoder in the context of non-energy conserving upmix rules |
US20080002842A1 (en) * | 2005-04-15 | 2008-01-03 | Fraunhofer-Geselschaft zur Forderung der angewandten Forschung e.V. | Apparatus and method for generating multi-channel synthesizer control signal and apparatus and method for multi-channel synthesizing |
US20080052089A1 (en) * | 2004-06-14 | 2008-02-28 | Matsushita Electric Industrial Co., Ltd. | Acoustic Signal Encoding Device and Acoustic Signal Decoding Device |
US20080097750A1 (en) * | 2005-06-03 | 2008-04-24 | Dolby Laboratories Licensing Corporation | Channel reconfiguration with side information |
US20080130904A1 (en) * | 2004-11-30 | 2008-06-05 | Agere Systems Inc. | Parametric Coding Of Spatial Audio With Object-Based Side Information |
US7391877B1 (en) * | 2003-03-31 | 2008-06-24 | United States Of America As Represented By The Secretary Of The Air Force | Spatial processor for enhanced performance in multi-talker speech displays |
US20080195397A1 (en) * | 2005-03-30 | 2008-08-14 | Koninklijke Philips Electronics, N.V. | Scalable Multi-Channel Audio Coding |
US20080192941A1 (en) * | 2006-12-07 | 2008-08-14 | Lg Electronics, Inc. | Method and an Apparatus for Decoding an Audio Signal |
US20090041265A1 (en) * | 2007-08-06 | 2009-02-12 | Katsutoshi Kubo | Sound signal processing device, sound signal processing method, sound signal processing program, storage medium, and display device |
US20090110203A1 (en) * | 2006-03-28 | 2009-04-30 | Anisse Taleb | Method and arrangement for a decoder for multi-channel surround sound |
US7536021B2 (en) * | 1997-09-16 | 2009-05-19 | Dolby Laboratories Licensing Corporation | Utilization of filtering effects in stereo headphone devices to enhance spatialization of source around a listener |
US20090129601A1 (en) * | 2006-01-09 | 2009-05-21 | Pasi Ojala | Controlling the Decoding of Binaural Audio Signals |
US7634092B2 (en) * | 2004-10-14 | 2009-12-15 | Dolby Laboratories Licensing Corporation | Head related transfer functions for panned stereo audio content |
US7668712B2 (en) * | 2004-03-31 | 2010-02-23 | Microsoft Corporation | Audio encoding and decoding with intra frames and adaptive forward error correction |
US7720230B2 (en) * | 2004-10-20 | 2010-05-18 | Agere Systems, Inc. | Individual channel shaping for BCC schemes and the like |
US7761304B2 (en) * | 2004-11-30 | 2010-07-20 | Agere Systems Inc. | Synchronizing parametric coding of spatial audio with externally provided downmix |
US7773756B2 (en) * | 1996-09-19 | 2010-08-10 | Terry D. Beard | Multichannel spectral mapping audio encoding apparatus and method with dynamically varying mapping coefficients |
US7797163B2 (en) * | 2006-08-18 | 2010-09-14 | Lg Electronics Inc. | Apparatus for processing media signal and method thereof |
US7880748B1 (en) * | 2005-08-17 | 2011-02-01 | Apple Inc. | Audio view using 3-dimensional plot |
US20110085669A1 (en) * | 2005-10-20 | 2011-04-14 | Lg Electronics, Inc. | Method for Encoding and Decoding Multi-Channel Audio Signal and Apparatus Thereof |
US7961889B2 (en) * | 2004-12-01 | 2011-06-14 | Samsung Electronics Co., Ltd. | Apparatus and method for processing multi-channel audio signal using space information |
US7979282B2 (en) * | 2006-09-29 | 2011-07-12 | Lg Electronics Inc. | Methods and apparatuses for encoding and decoding object-based audio signals |
US8108220B2 (en) * | 2000-03-02 | 2012-01-31 | Akiba Electronics Institute Llc | Techniques for accommodating primary content (pure voice) audio and secondary content remaining audio capability in the digital audio production process |
US8116459B2 (en) * | 2006-03-28 | 2012-02-14 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Enhanced method for signal shaping in multi-channel audio reconstruction |
US8150042B2 (en) * | 2004-07-14 | 2012-04-03 | Koninklijke Philips Electronics N.V. | Method, device, encoder apparatus, decoder apparatus and audio system |
US8185403B2 (en) * | 2005-06-30 | 2012-05-22 | Lg Electronics Inc. | Method and apparatus for encoding and decoding an audio signal |
US8189682B2 (en) * | 2008-03-27 | 2012-05-29 | Oki Electric Industry Co., Ltd. | Decoding system and method for error correction with side information and correlation updater |
US8255211B2 (en) * | 2004-08-25 | 2012-08-28 | Dolby Laboratories Licensing Corporation | Temporal envelope shaping for spatial audio coding using frequency domain wiener filtering |
US8488819B2 (en) * | 2006-01-19 | 2013-07-16 | Lg Electronics Inc. | Method and apparatus for processing a media signal |
Family Cites Families (90)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE4217276C1 (en) | 1992-05-25 | 1993-04-08 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung Ev, 8000 Muenchen, De | |
DE4236989C2 (en) | 1992-11-02 | 1994-11-17 | Fraunhofer Ges Forschung | Method for transmitting and / or storing digital signals of multiple channels |
TW263646B (en) | 1993-08-26 | 1995-11-21 | Nat Science Committee | Synchronizing method for multimedia signal |
JPH07248255A (en) | 1994-03-09 | 1995-09-26 | Sharp Corp | Method and apparatus for forming stereophonic image |
JPH07288900A (en) * | 1994-04-19 | 1995-10-31 | Matsushita Electric Ind Co Ltd | Sound field reproducing device |
CA2189126C (en) | 1994-05-11 | 2001-05-01 | Jonathan S. Abel | Three-dimensional virtual audio display employing reduced complexity imaging filters |
GB9417185D0 (en) | 1994-08-25 | 1994-10-12 | Adaptive Audio Ltd | Sounds recording and reproduction systems |
JP3395807B2 (en) * | 1994-09-07 | 2003-04-14 | 日本電信電話株式会社 | Stereo sound reproducer |
JPH0884400A (en) | 1994-09-12 | 1996-03-26 | Sanyo Electric Co Ltd | Sound image controller |
JPH08123494A (en) | 1994-10-28 | 1996-05-17 | Mitsubishi Electric Corp | Speech encoding device, speech decoding device, speech encoding and decoding method, and phase amplitude characteristic derivation device usable for same |
JPH0974446A (en) * | 1995-03-01 | 1997-03-18 | Nippon Telegr & Teleph Corp <Ntt> | Voice communication controller |
IT1281001B1 (en) | 1995-10-27 | 1998-02-11 | Cselt Centro Studi Lab Telecom | PROCEDURE AND EQUIPMENT FOR CODING, HANDLING AND DECODING AUDIO SIGNALS. |
US5956674A (en) | 1995-12-01 | 1999-09-21 | Digital Theater Systems, Inc. | Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels |
JP3088319B2 (en) | 1996-02-07 | 2000-09-18 | 松下電器産業株式会社 | Decoding device and decoding method |
JPH09224300A (en) | 1996-02-16 | 1997-08-26 | Sanyo Electric Co Ltd | Method and device for correcting sound image position |
JP3483086B2 (en) | 1996-03-22 | 2004-01-06 | 日本電信電話株式会社 | Audio teleconferencing equipment |
US5970152A (en) | 1996-04-30 | 1999-10-19 | Srs Labs, Inc. | Audio enhancement system for use in a surround sound environment |
TW429700B (en) | 1997-02-26 | 2001-04-11 | Sony Corp | Information encoding method and apparatus, information decoding method and apparatus and information recording medium |
US6449368B1 (en) * | 1997-03-14 | 2002-09-10 | Dolby Laboratories Licensing Corporation | Multidirectional audio decoding |
JP3594281B2 (en) | 1997-04-30 | 2004-11-24 | 株式会社河合楽器製作所 | Stereo expansion device and sound field expansion device |
JPH1132400A (en) | 1997-07-14 | 1999-02-02 | Matsushita Electric Ind Co Ltd | Digital signal reproducing device |
US6081783A (en) | 1997-11-14 | 2000-06-27 | Cirrus Logic, Inc. | Dual processor digital audio decoder with shared memory data transfer and task partitioning for decompressing compressed audio data, and systems and methods using the same |
US6414290B1 (en) * | 1998-03-19 | 2002-07-02 | Graphic Packaging Corporation | Patterned microwave susceptor |
DE19846576C2 (en) | 1998-10-09 | 2001-03-08 | Aeg Niederspannungstech Gmbh | Sealable sealing device |
DE19847689B4 (en) | 1998-10-15 | 2013-07-11 | Samsung Electronics Co., Ltd. | Apparatus and method for three-dimensional sound reproduction |
JP3346556B2 (en) | 1998-11-16 | 2002-11-18 | 日本ビクター株式会社 | Audio encoding method and audio decoding method |
KR100416757B1 (en) | 1999-06-10 | 2004-01-31 | 삼성전자주식회사 | Multi-channel audio reproduction apparatus and method for loud-speaker reproduction |
US6931370B1 (en) | 1999-11-02 | 2005-08-16 | Digital Theater Systems, Inc. | System and method for providing interactive audio in a multi-channel audio environment |
US6973130B1 (en) * | 2000-04-25 | 2005-12-06 | Wee Susie J | Compressed video signal including information for independently coded regions |
TW468182B (en) | 2000-05-03 | 2001-12-11 | Ind Tech Res Inst | Method and device for adjusting, recording and playing multimedia signals |
JP2001359197A (en) * | 2000-06-13 | 2001-12-26 | Victor Co Of Japan Ltd | Method and device for generating sound image localizing signal |
JP3576936B2 (en) | 2000-07-21 | 2004-10-13 | 株式会社ケンウッド | Frequency interpolation device, frequency interpolation method, and recording medium |
JP4645869B2 (en) | 2000-08-02 | 2011-03-09 | ソニー株式会社 | DIGITAL SIGNAL PROCESSING METHOD, LEARNING METHOD, DEVICE THEREOF, AND PROGRAM STORAGE MEDIUM |
EP1211857A1 (en) | 2000-12-04 | 2002-06-05 | STMicroelectronics N.V. | Process and device of successive value estimations of numerical symbols, in particular for the equalization of a data communication channel of information in mobile telephony |
WO2004019656A2 (en) | 2001-02-07 | 2004-03-04 | Dolby Laboratories Licensing Corporation | Audio channel spatial translation |
JP3566220B2 (en) | 2001-03-09 | 2004-09-15 | 三菱電機株式会社 | Speech coding apparatus, speech coding method, speech decoding apparatus, and speech decoding method |
US6504496B1 (en) * | 2001-04-10 | 2003-01-07 | Cirrus Logic, Inc. | Systems and methods for decoding compressed data |
US20030007648A1 (en) | 2001-04-27 | 2003-01-09 | Christopher Currell | Virtual audio system and techniques |
US20030035553A1 (en) | 2001-08-10 | 2003-02-20 | Frank Baumgarte | Backwards-compatible perceptual coding of spatial cues |
JP2003009296A (en) | 2001-06-22 | 2003-01-10 | Matsushita Electric Ind Co Ltd | Acoustic processing unit and acoustic processing method |
SE0202159D0 (en) | 2001-07-10 | 2002-07-09 | Coding Technologies Sweden Ab | Efficientand scalable parametric stereo coding for low bitrate applications |
JP2003111198A (en) | 2001-10-01 | 2003-04-11 | Sony Corp | Voice signal processing method and voice reproducing system |
EP1315148A1 (en) | 2001-11-17 | 2003-05-28 | Deutsche Thomson-Brandt Gmbh | Determination of the presence of ancillary data in an audio bitstream |
TWI230024B (en) | 2001-12-18 | 2005-03-21 | Dolby Lab Licensing Corp | Method and audio apparatus for improving spatial perception of multiple sound channels when reproduced by two loudspeakers |
DE60323331D1 (en) | 2002-01-30 | 2008-10-16 | Matsushita Electric Ind Co Ltd | METHOD AND DEVICE FOR AUDIO ENCODING AND DECODING |
EP1341160A1 (en) | 2002-03-01 | 2003-09-03 | Deutsche Thomson-Brandt Gmbh | Method and apparatus for encoding and for decoding a digital information signal |
US7707287B2 (en) | 2002-03-22 | 2010-04-27 | F5 Networks, Inc. | Virtual host acceleration system |
WO2003085643A1 (en) | 2002-04-10 | 2003-10-16 | Koninklijke Philips Electronics N.V. | Coding of stereo signals |
WO2003090206A1 (en) * | 2002-04-22 | 2003-10-30 | Koninklijke Philips Electronics N.V. | Signal synthesizing |
BRPI0305434B1 (en) | 2002-07-12 | 2017-06-27 | Koninklijke Philips Electronics N.V. | Methods and arrangements for encoding and decoding a multichannel audio signal, and multichannel audio coded signal |
AU2003281128A1 (en) | 2002-07-16 | 2004-02-02 | Koninklijke Philips Electronics N.V. | Audio coding |
CN1328707C (en) | 2002-07-19 | 2007-07-25 | 日本电气株式会社 | Audio decoding device, decoding method, and program |
USRE43273E1 (en) * | 2002-09-23 | 2012-03-27 | Koninklijke Philips Electronics N.V. | Generation of a sound signal |
CN1689070A (en) | 2002-10-14 | 2005-10-26 | 皇家飞利浦电子股份有限公司 | Signal filtering |
DE60312553T2 (en) | 2002-10-14 | 2007-11-29 | Thomson Licensing | PROCESS FOR CODING AND DECODING THE WIDTH OF A SOUND SOURCE IN AN AUDIOSCENE |
AU2003269551A1 (en) | 2002-10-15 | 2004-05-04 | Electronics And Telecommunications Research Institute | Method for generating and consuming 3d audio scene with extended spatiality of sound source |
JP4393383B2 (en) | 2002-10-15 | 2010-01-06 | 韓國電子通信研究院 | Audio signal adaptive conversion apparatus according to user preference and method thereof |
US8139797B2 (en) * | 2002-12-03 | 2012-03-20 | Bose Corporation | Directional electroacoustical transducing |
US6829925B2 (en) | 2002-12-20 | 2004-12-14 | The Goodyear Tire & Rubber Company | Apparatus and method for monitoring a condition of a tire |
KR100917464B1 (en) | 2003-03-07 | 2009-09-14 | 삼성전자주식회사 | Method and apparatus for encoding/decoding digital data using bandwidth extension technology |
JP4196274B2 (en) | 2003-08-11 | 2008-12-17 | ソニー株式会社 | Image signal processing apparatus and method, program, and recording medium |
CN1253464C (en) | 2003-08-13 | 2006-04-26 | 中国科学院昆明植物研究所 | Ansi glycoside compound and its medicinal composition, preparation and use |
US20050063613A1 (en) | 2003-09-24 | 2005-03-24 | Kevin Casey | Network based system and method to process images |
JP4966013B2 (en) | 2003-10-30 | 2012-07-04 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Encode or decode audio signals |
US7949141B2 (en) | 2003-11-12 | 2011-05-24 | Dolby Laboratories Licensing Corporation | Processing audio signals with head related transfer function filters and a reverberator |
CN100584037C (en) | 2004-01-05 | 2010-01-20 | 皇家飞利浦电子股份有限公司 | Flicker-free adaptive thresholding for ambient light derived from video content mapped through unrendered color space |
ES2687432T3 (en) | 2004-01-05 | 2018-10-25 | Tp Vision Holding B.V. | Ambient light derived from video content through mapping transformations through a non-rendered color space |
US8983834B2 (en) | 2004-03-01 | 2015-03-17 | Dolby Laboratories Licensing Corporation | Multichannel audio coding |
US7805313B2 (en) * | 2004-03-04 | 2010-09-28 | Agere Systems Inc. | Frequency-based coding of channels in parametric multi-channel coding systems |
CN1947172B (en) | 2004-04-05 | 2011-08-03 | 皇家飞利浦电子股份有限公司 | Method, device, encoder apparatus, decoder apparatus and frequency system |
TWI253625B (en) | 2004-04-06 | 2006-04-21 | I-Shun Huang | Signal-processing system and method thereof |
US20050276430A1 (en) | 2004-05-28 | 2005-12-15 | Microsoft Corporation | Fast headphone virtualization |
US7283065B2 (en) * | 2004-06-02 | 2007-10-16 | Research In Motion Limited | Handheld electronic device with text disambiguation |
KR100636144B1 (en) * | 2004-06-04 | 2006-10-18 | 삼성전자주식회사 | Apparatus and method for encoding/decoding audio signal |
JP4594662B2 (en) | 2004-06-29 | 2010-12-08 | ソニー株式会社 | Sound image localization device |
WO2006003813A1 (en) * | 2004-07-02 | 2006-01-12 | Matsushita Electric Industrial Co., Ltd. | Audio encoding and decoding apparatus |
KR20060003444A (en) | 2004-07-06 | 2006-01-11 | 삼성전자주식회사 | Cross-talk canceller device and method in mobile telephony |
KR100773539B1 (en) | 2004-07-14 | 2007-11-05 | 삼성전자주식회사 | Multi channel audio data encoding/decoding method and apparatus |
JP4641751B2 (en) * | 2004-07-23 | 2011-03-02 | ローム株式会社 | Peak hold circuit, motor drive control circuit including the same, and motor device including the same |
TWI393121B (en) | 2004-08-25 | 2013-04-11 | Dolby Lab Licensing Corp | Method and apparatus for processing a set of n audio signals, and computer program associated therewith |
DE102004042819A1 (en) * | 2004-09-03 | 2006-03-23 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for generating a coded multi-channel signal and apparatus and method for decoding a coded multi-channel signal |
KR20060022968A (en) | 2004-09-08 | 2006-03-13 | 삼성전자주식회사 | Sound reproducing apparatus and sound reproducing method |
US8027494B2 (en) * | 2004-11-22 | 2011-09-27 | Mitsubishi Electric Corporation | Acoustic image creation system and program therefor |
US7903824B2 (en) * | 2005-01-10 | 2011-03-08 | Agere Systems Inc. | Compact side information for parametric coding of spatial audio |
US7573912B2 (en) | 2005-02-22 | 2009-08-11 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschunng E.V. | Near-transparent or transparent multi-channel encoder/decoder scheme |
EP1906706B1 (en) | 2005-07-15 | 2009-11-25 | Panasonic Corporation | Audio decoder |
KR101370365B1 (en) * | 2005-09-13 | 2014-03-05 | 코닌클리케 필립스 엔.브이. | A method of and a device for generating 3D sound |
CN104681030B (en) | 2006-02-07 | 2018-02-27 | Lg电子株式会社 | Apparatus and method for encoding/decoding signal |
JP4778828B2 (en) | 2006-04-14 | 2011-09-21 | 矢崎総業株式会社 | Electrical junction box |
US7876904B2 (en) * | 2006-07-08 | 2011-01-25 | Nokia Corporation | Dynamic decoding of binaural audio signals |
-
2007
- 2007-01-19 WO PCT/KR2007/000349 patent/WO2007083959A1/en active Application Filing
- 2007-01-19 EP EP07708529.8A patent/EP1979898B1/en active Active
- 2007-01-19 EP EP07708534.8A patent/EP1974348B1/en active Active
- 2007-01-19 US US12/161,329 patent/US8521313B2/en active Active
- 2007-01-19 CA CA2636494A patent/CA2636494C/en active Active
- 2007-01-19 JP JP2008551193A patent/JP4814343B2/en active Active
- 2007-01-19 US US12/161,560 patent/US20090028344A1/en not_active Abandoned
- 2007-01-19 KR KR1020087005977A patent/KR100953640B1/en active IP Right Grant
- 2007-01-19 ES ES07701036.1T patent/ES2446245T3/en active Active
- 2007-01-19 US US12/161,558 patent/US8411869B2/en active Active
- 2007-01-19 WO PCT/KR2007/000345 patent/WO2007083955A1/en active Application Filing
- 2007-01-19 KR KR1020087005982A patent/KR100953645B1/en active IP Right Grant
- 2007-01-19 TW TW096102405A patent/TWI329461B/en active
- 2007-01-19 WO PCT/KR2007/000340 patent/WO2007083952A1/en active Application Filing
- 2007-01-19 JP JP2008551196A patent/JP4787331B2/en active Active
- 2007-01-19 JP JP2008551199A patent/JP4695197B2/en active Active
- 2007-01-19 EP EP07701036.1A patent/EP1974345B1/en active Active
- 2007-01-19 US US12/161,337 patent/US8351611B2/en active Active
- 2007-01-19 WO PCT/KR2007/000342 patent/WO2007083953A1/en active Application Filing
- 2007-01-19 TW TW096102406A patent/TWI329462B/en active
- 2007-01-19 EP EP07701033.8A patent/EP1979897B1/en active Active
- 2007-01-19 KR KR1020087021417A patent/KR20080086548A/en not_active Application Discontinuation
- 2007-01-19 TW TW98105568A patent/TWI469133B/en not_active IP Right Cessation
- 2007-01-19 JP JP2008551194A patent/JP4814344B2/en active Active
- 2007-01-19 WO PCT/KR2007/000346 patent/WO2007083956A1/en active Application Filing
- 2007-01-19 KR KR1020087005980A patent/KR100953643B1/en active IP Right Grant
- 2007-01-19 KR KR1020087005978A patent/KR100953641B1/en active IP Right Grant
- 2007-01-19 TW TW096102403A patent/TWI315864B/en active
- 2007-01-19 EP EP07701037.9A patent/EP1974346B1/en active Active
- 2007-01-19 ES ES07708531.4T patent/ES2513265T3/en active Active
- 2007-01-19 KR KR1020087005979A patent/KR100953642B1/en active IP Right Grant
- 2007-01-19 AU AU2007206195A patent/AU2007206195B2/en active Active
- 2007-01-19 US US12/161,334 patent/US8208641B2/en active Active
- 2007-01-19 TW TW096102407A patent/TWI333642B/en active
- 2007-01-19 KR KR1020070006281A patent/KR20070077134A/en not_active Application Discontinuation
- 2007-01-19 JP JP2008551200A patent/JP4806031B2/en active Active
- 2007-01-19 ES ES07708529.8T patent/ES2496571T3/en active Active
- 2007-01-19 EP EP07708531.4A patent/EP1974347B1/en active Active
- 2007-01-19 US US12/161,563 patent/US8488819B2/en active Active
- 2007-01-19 JP JP2008551195A patent/JP4801174B2/en active Active
- 2007-01-19 WO PCT/KR2007/000350 patent/WO2007083960A1/en active Application Filing
- 2007-01-19 TW TW096102411A patent/TWI333386B/en active
- 2007-01-19 KR KR1020087005981A patent/KR100953644B1/en active IP Right Grant
- 2007-01-19 TW TW096102410A patent/TWI344638B/en active
- 2007-01-19 BR BRPI0707136-1A patent/BRPI0707136A2/en not_active IP Right Cessation
-
2009
- 2009-06-05 HK HK09105049.1A patent/HK1127433A1/en not_active IP Right Cessation
Patent Citations (103)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5166685A (en) * | 1990-09-04 | 1992-11-24 | Motorola, Inc. | Automatic selection of external multiplexer channels by an A/D converter integrated circuit |
US5632005A (en) * | 1991-01-08 | 1997-05-20 | Ray Milton Dolby | Encoder/decoder for multidimensional sound fields |
US5561736A (en) * | 1993-06-04 | 1996-10-01 | International Business Machines Corporation | Three dimensional speech synthesis |
US5524054A (en) * | 1993-06-22 | 1996-06-04 | Deutsche Thomson-Brandt Gmbh | Method for generating a multi-channel audio decoder matrix |
US5579396A (en) * | 1993-07-30 | 1996-11-26 | Victor Company Of Japan, Ltd. | Surround signal processing apparatus |
US6118875A (en) * | 1994-02-25 | 2000-09-12 | Moeller; Henrik | Binaural synthesis, head-related transfer functions, and uses thereof |
US5703584A (en) * | 1994-08-22 | 1997-12-30 | Adaptec, Inc. | Analog data acquisition system |
US6072877A (en) * | 1994-09-09 | 2000-06-06 | Aureal Semiconductor, Inc. | Three-dimensional virtual audio display employing reduced complexity imaging filters |
US5668924A (en) * | 1995-01-18 | 1997-09-16 | Olympus Optical Co. Ltd. | Digital sound recording and reproduction device using a coding technique to compress data for reduction of memory requirements |
US7773756B2 (en) * | 1996-09-19 | 2010-08-10 | Terry D. Beard | Multichannel spectral mapping audio encoding apparatus and method with dynamically varying mapping coefficients |
US5886988A (en) * | 1996-10-23 | 1999-03-23 | Arraycomm, Inc. | Channel assignment and call admission control for spatial division multiple access communication systems |
US6711266B1 (en) * | 1997-02-07 | 2004-03-23 | Bose Corporation | Surround sound channel encoding and decoding |
US6721425B1 (en) * | 1997-02-07 | 2004-04-13 | Bose Corporation | Sound signal mixing |
US6307941B1 (en) * | 1997-07-15 | 2001-10-23 | Desper Products, Inc. | System and method for localization of virtual sound |
US5890125A (en) * | 1997-07-16 | 1999-03-30 | Dolby Laboratories Licensing Corporation | Method and apparatus for encoding and decoding multiple audio channels at low bit rates using adaptive selection of encoding method |
US7536021B2 (en) * | 1997-09-16 | 2009-05-19 | Dolby Laboratories Licensing Corporation | Utilization of filtering effects in stereo headphone devices to enhance spatialization of source around a listener |
US20060251276A1 (en) * | 1997-11-14 | 2006-11-09 | Jiashu Chen | Generating 3D audio using a regularized HRTF/HRIR filter |
US6122619A (en) * | 1998-06-17 | 2000-09-19 | Lsi Logic Corporation | Audio decoder with programmable downmixing of MPEG/AC-3 and method therefor |
US6466913B1 (en) * | 1998-07-01 | 2002-10-15 | Ricoh Company, Ltd. | Method of determining a sound localization filter and a sound localization control system incorporating the filter |
US6574339B1 (en) * | 1998-10-20 | 2003-06-03 | Samsung Electronics Co., Ltd. | Three-dimensional sound reproducing apparatus for multiple listeners and method thereof |
US7085393B1 (en) * | 1998-11-13 | 2006-08-01 | Agere Systems Inc. | Method and apparatus for regularizing measured HRTF for smooth 3D digital audio |
US6611212B1 (en) * | 1999-04-07 | 2003-08-26 | Dolby Laboratories Licensing Corp. | Matrix improvements to lossless encoding and decoding |
US6795556B1 (en) * | 1999-05-29 | 2004-09-21 | Creative Technology, Ltd. | Method of modifying one or more original head related transfer functions |
US6226616B1 (en) * | 1999-06-21 | 2001-05-01 | Digital Theater Systems, Inc. | Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility |
US7177431B2 (en) * | 1999-07-09 | 2007-02-13 | Creative Technology, Ltd. | Dynamic decorrelator for audio signals |
US20060126851A1 (en) * | 1999-10-04 | 2006-06-15 | Yuen Thomas C | Acoustic correction apparatus |
US6633648B1 (en) * | 1999-11-12 | 2003-10-14 | Jerald L. Bauck | Loudspeaker array for enlarged sweet spot |
US20040071445A1 (en) * | 1999-12-23 | 2004-04-15 | Tarnoff Harry L. | Method and apparatus for synchronization of ancillary information in film conversion |
US20070183603A1 (en) * | 2000-01-17 | 2007-08-09 | Vast Audio Pty Ltd | Generation of customised three dimensional sound effects for individuals |
US20010031062A1 (en) * | 2000-02-02 | 2001-10-18 | Kenichi Terai | Headphone system |
US8108220B2 (en) * | 2000-03-02 | 2012-01-31 | Akiba Electronics Institute Llc | Techniques for accommodating primary content (pure voice) audio and secondary content remaining audio capability in the digital audio production process |
US7302068B2 (en) * | 2001-06-21 | 2007-11-27 | 1 . . .Limited | Loudspeaker |
US7260540B2 (en) * | 2001-11-14 | 2007-08-21 | Matsushita Electric Industrial Co., Ltd. | Encoding device, decoding device, and system thereof utilizing band expansion information |
US20040032960A1 (en) * | 2002-05-03 | 2004-02-19 | Griesinger David H. | Multichannel downmixing device |
US20040196770A1 (en) * | 2002-05-07 | 2004-10-07 | Keisuke Touyama | Coding method, coding device, decoding method, and decoding device |
US20030236583A1 (en) * | 2002-06-24 | 2003-12-25 | Frank Baumgarte | Hybrid multi-channel/cue coding/decoding of audio signals |
US7180964B2 (en) * | 2002-06-28 | 2007-02-20 | Advanced Micro Devices, Inc. | Constellation manipulation for frequency/phase error correction |
US20040049379A1 (en) * | 2002-09-04 | 2004-03-11 | Microsoft Corporation | Multi-channel audio encoding and decoding |
US20040111171A1 (en) * | 2002-10-28 | 2004-06-10 | Dae-Young Jang | Object-based three-dimensional audio system and method of controlling the same |
US20060072764A1 (en) * | 2002-11-20 | 2006-04-06 | Koninklijke Philips Electronics N.V. | Audio based data representation apparatus and method |
US20040138874A1 (en) * | 2003-01-09 | 2004-07-15 | Samu Kaajas | Audio signal processing |
US7519530B2 (en) * | 2003-01-09 | 2009-04-14 | Nokia Corporation | Audio signal processing |
US7391877B1 (en) * | 2003-03-31 | 2008-06-24 | United States Of America As Represented By The Secretary Of The Air Force | Spatial processor for enhanced performance in multi-talker speech displays |
US20050074127A1 (en) * | 2003-10-02 | 2005-04-07 | Jurgen Herre | Compatible multi-channel coding/decoding |
US20050089181A1 (en) * | 2003-10-27 | 2005-04-28 | Polk Matthew S.Jr. | Multi-channel audio surround sound from front located loudspeakers |
US20050117762A1 (en) * | 2003-11-04 | 2005-06-02 | Atsuhiro Sakurai | Binaural sound localization using a formant-type cascade of resonators and anti-resonators |
US20070165886A1 (en) * | 2003-11-17 | 2007-07-19 | Richard Topliss | Louderspeaker |
US20050135643A1 (en) * | 2003-12-17 | 2005-06-23 | Joon-Hyun Lee | Apparatus and method of reproducing virtual sound |
US20050157883A1 (en) * | 2004-01-20 | 2005-07-21 | Jurgen Herre | Apparatus and method for constructing a multi-channel output signal or for generating a downmix signal |
US20050180579A1 (en) * | 2004-02-12 | 2005-08-18 | Frank Baumgarte | Late reverberation-based synthesis of auditory scenes |
US20050179701A1 (en) * | 2004-02-13 | 2005-08-18 | Jahnke Steven R. | Dynamic sound source and listener position based audio rendering |
US20070162278A1 (en) * | 2004-02-25 | 2007-07-12 | Matsushita Electric Industrial Co., Ltd. | Audio encoder and audio decoder |
US7668712B2 (en) * | 2004-03-31 | 2010-02-23 | Microsoft Corporation | Audio encoding and decoding with intra frames and adaptive forward error correction |
US20070258607A1 (en) * | 2004-04-16 | 2007-11-08 | Heiko Purnhagen | Method for representing multi-channel audio signals |
US20050273322A1 (en) * | 2004-06-04 | 2005-12-08 | Hyuck-Jae Lee | Audio signal encoding and decoding apparatus |
US20050273324A1 (en) * | 2004-06-08 | 2005-12-08 | Expamedia, Inc. | System for providing audio data and providing method thereof |
US20080052089A1 (en) * | 2004-06-14 | 2008-02-28 | Matsushita Electric Industrial Co., Ltd. | Acoustic Signal Encoding Device and Acoustic Signal Decoding Device |
US20060004583A1 (en) * | 2004-06-30 | 2006-01-05 | Juergen Herre | Multi-channel synthesizer and method for generating a multi-channel output signal |
US20060002572A1 (en) * | 2004-07-01 | 2006-01-05 | Smithers Michael J | Method for correcting metadata affecting the playback loudness and dynamic range of audio information |
US20060008094A1 (en) * | 2004-07-06 | 2006-01-12 | Jui-Jung Huang | Wireless multi-channel audio system |
US20060009225A1 (en) * | 2004-07-09 | 2006-01-12 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | Apparatus and method for generating a multi-channel output signal |
US8150042B2 (en) * | 2004-07-14 | 2012-04-03 | Koninklijke Philips Electronics N.V. | Method, device, encoder apparatus, decoder apparatus and audio system |
US8255211B2 (en) * | 2004-08-25 | 2012-08-28 | Dolby Laboratories Licensing Corporation | Temporal envelope shaping for spatial audio coding using frequency domain wiener filtering |
US7634092B2 (en) * | 2004-10-14 | 2009-12-15 | Dolby Laboratories Licensing Corporation | Head related transfer functions for panned stereo audio content |
US7720230B2 (en) * | 2004-10-20 | 2010-05-18 | Agere Systems, Inc. | Individual channel shaping for BCC schemes and the like |
US20060133618A1 (en) * | 2004-11-02 | 2006-06-22 | Lars Villemoes | Stereo compatible multi-channel audio coding |
US20060115100A1 (en) * | 2004-11-30 | 2006-06-01 | Christof Faller | Parametric coding of spatial audio with cues based on transmitted channels |
US7761304B2 (en) * | 2004-11-30 | 2010-07-20 | Agere Systems Inc. | Synchronizing parametric coding of spatial audio with externally provided downmix |
US7787631B2 (en) * | 2004-11-30 | 2010-08-31 | Agere Systems Inc. | Parametric coding of spatial audio with cues based on transmitted channels |
US20080130904A1 (en) * | 2004-11-30 | 2008-06-05 | Agere Systems Inc. | Parametric Coding Of Spatial Audio With Object-Based Side Information |
US7961889B2 (en) * | 2004-12-01 | 2011-06-14 | Samsung Electronics Co., Ltd. | Apparatus and method for processing multi-channel audio signal using space information |
US20060195981A1 (en) * | 2005-03-02 | 2006-09-07 | Hydro-Industries Tynat Ltd. | Freestanding combination sink and hose reel workstation |
US20060198527A1 (en) * | 2005-03-03 | 2006-09-07 | Ingyu Chun | Method and apparatus to generate stereo sound for two-channel headphones |
US20080195397A1 (en) * | 2005-03-30 | 2008-08-14 | Koninklijke Philips Electronics, N.V. | Scalable Multi-Channel Audio Coding |
US20080002842A1 (en) * | 2005-04-15 | 2008-01-03 | Fraunhofer-Geselschaft zur Forderung der angewandten Forschung e.V. | Apparatus and method for generating multi-channel synthesizer control signal and apparatus and method for multi-channel synthesizing |
US20060233380A1 (en) * | 2005-04-15 | 2006-10-19 | FRAUNHOFER- GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG e.V. | Multi-channel hierarchical audio coding with compact side information |
US20060233379A1 (en) * | 2005-04-15 | 2006-10-19 | Coding Technologies, AB | Adaptive residual audio coding |
US20060239473A1 (en) * | 2005-04-15 | 2006-10-26 | Coding Technologies Ab | Envelope shaping of decorrelated signals |
US20080097750A1 (en) * | 2005-06-03 | 2008-04-24 | Dolby Laboratories Licensing Corporation | Channel reconfiguration with side information |
US8185403B2 (en) * | 2005-06-30 | 2012-05-22 | Lg Electronics Inc. | Method and apparatus for encoding and decoding an audio signal |
US7880748B1 (en) * | 2005-08-17 | 2011-02-01 | Apple Inc. | Audio view using 3-dimensional plot |
US20070203697A1 (en) * | 2005-08-30 | 2007-08-30 | Hee Suk Pang | Time slot position coding of multiple frame types |
US20070133831A1 (en) * | 2005-09-22 | 2007-06-14 | Samsung Electronics Co., Ltd. | Apparatus and method of reproducing virtual sound of two channels |
US20110085669A1 (en) * | 2005-10-20 | 2011-04-14 | Lg Electronics, Inc. | Method for Encoding and Decoding Multi-Channel Audio Signal and Apparatus Thereof |
WO2007068243A1 (en) * | 2005-12-16 | 2007-06-21 | Widex A/S | Method and system for surveillance of a wireless connection in a hearing aid fitting system |
US20070160218A1 (en) * | 2006-01-09 | 2007-07-12 | Nokia Corporation | Decoding of binaural audio signals |
US20090129601A1 (en) * | 2006-01-09 | 2009-05-21 | Pasi Ojala | Controlling the Decoding of Binaural Audio Signals |
US20070233296A1 (en) * | 2006-01-11 | 2007-10-04 | Samsung Electronics Co., Ltd. | Method, medium, and apparatus with scalable channel decoding |
US8488819B2 (en) * | 2006-01-19 | 2013-07-16 | Lg Electronics Inc. | Method and apparatus for processing a media signal |
US20070172071A1 (en) * | 2006-01-20 | 2007-07-26 | Microsoft Corporation | Complex transforms for multi-channel audio |
US20070223709A1 (en) * | 2006-03-06 | 2007-09-27 | Samsung Electronics Co., Ltd. | Method, medium, and system generating a stereo signal |
US20070223708A1 (en) * | 2006-03-24 | 2007-09-27 | Lars Villemoes | Generation of spatial downmixes from parametric representations of multi channel signals |
US8116459B2 (en) * | 2006-03-28 | 2012-02-14 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Enhanced method for signal shaping in multi-channel audio reconstruction |
US20090110203A1 (en) * | 2006-03-28 | 2009-04-30 | Anisse Taleb | Method and arrangement for a decoder for multi-channel surround sound |
US20070280485A1 (en) * | 2006-06-02 | 2007-12-06 | Lars Villemoes | Binaural multi-channel decoder in the context of non-energy conserving upmix rules |
US7797163B2 (en) * | 2006-08-18 | 2010-09-14 | Lg Electronics Inc. | Apparatus for processing media signal and method thereof |
US7979282B2 (en) * | 2006-09-29 | 2011-07-12 | Lg Electronics Inc. | Methods and apparatuses for encoding and decoding object-based audio signals |
US7987096B2 (en) * | 2006-09-29 | 2011-07-26 | Lg Electronics Inc. | Methods and apparatuses for encoding and decoding object-based audio signals |
US20080199026A1 (en) * | 2006-12-07 | 2008-08-21 | Lg Electronics, Inc. | Method and an Apparatus for Decoding an Audio Signal |
US20080192941A1 (en) * | 2006-12-07 | 2008-08-14 | Lg Electronics, Inc. | Method and an Apparatus for Decoding an Audio Signal |
US20090041265A1 (en) * | 2007-08-06 | 2009-02-12 | Katsutoshi Kubo | Sound signal processing device, sound signal processing method, sound signal processing program, storage medium, and display device |
US8150066B2 (en) * | 2007-08-06 | 2012-04-03 | Sharp Kabushiki Kaisha | Sound signal processing device, sound signal processing method, sound signal processing program, storage medium, and display device |
US8189682B2 (en) * | 2008-03-27 | 2012-05-29 | Oki Electric Industry Co., Ltd. | Decoding system and method for error correction with side information and correlation updater |
Non-Patent Citations (2)
Title |
---|
Breebaart et al, Parametric Coding of stereo audio, Eurasip 2005 * |
Savioja Lauri, Modeling Techniques for Virtual Acoustics, Thesis, August 2000 * |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100310081A1 (en) * | 2009-06-08 | 2010-12-09 | Mstar Semiconductor, Inc. | Multi-channel Audio Signal Decoding Method and Device |
TWI404050B (en) * | 2009-06-08 | 2013-08-01 | Mstar Semiconductor Inc | Multi-channel audio signal decoding method and device |
US8503684B2 (en) * | 2009-06-08 | 2013-08-06 | Mstar Semiconductor, Inc. | Multi-channel audio signal decoding method and device |
US8515771B2 (en) | 2009-09-01 | 2013-08-20 | Panasonic Corporation | Identifying an encoding format of an encoded voice signal |
US9060236B2 (en) * | 2009-10-20 | 2015-06-16 | Dolby International Ab | Apparatus for providing an upmix signal representation on the basis of a downmix signal representation, apparatus for providing a bitstream representing a multi-channel audio signal, methods, computer program and bitstream using a distortion control signaling |
US9093080B2 (en) | 2010-06-09 | 2015-07-28 | Panasonic Intellectual Property Corporation Of America | Bandwidth extension method, bandwidth extension apparatus, program, integrated circuit, and audio decoding apparatus |
US9799342B2 (en) | 2010-06-09 | 2017-10-24 | Panasonic Intellectual Property Corporation Of America | Bandwidth extension method, bandwidth extension apparatus, program, integrated circuit, and audio decoding apparatus |
US10566001B2 (en) | 2010-06-09 | 2020-02-18 | Panasonic Intellectual Property Corporation Of America | Bandwidth extension method, bandwidth extension apparatus, program, integrated circuit, and audio decoding apparatus |
US11341977B2 (en) | 2010-06-09 | 2022-05-24 | Panasonic Intellectual Property Corporation Of America | Bandwidth extension method, bandwidth extension apparatus, program, integrated circuit, and audio decoding apparatus |
US11749289B2 (en) | 2010-06-09 | 2023-09-05 | Panasonic Intellectual Property Corporation Of America | Bandwidth extension method, bandwidth extension apparatus, program, integrated circuit, and audio decoding apparatus |
US20120092566A1 (en) * | 2010-10-19 | 2012-04-19 | Samsung Electronics Co., Ltd. | Image processing apparatus, sound processing method used for image processing apparatus, and sound processing apparatus |
US10142763B2 (en) | 2013-11-27 | 2018-11-27 | Dolby Laboratories Licensing Corporation | Audio signal processing |
Also Published As
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8208641B2 (en) | Method and apparatus for processing a media signal | |
MX2008008308A (en) | Method and apparatus for processing a media signal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PANG, HEE SUK;KIM, DONG SOO;LIM, JAE HYUN;AND OTHERS;REEL/FRAME:021282/0371 Effective date: 20080710 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |