KR102380232B1 - Method and apparatus for 3D sound reproducing - Google Patents

Abstract

An audio signal rendering method is disclosed. An audio signal rendering method includes receiving a multi-channel signal including a height input channel signal and a rendering type, wherein the multi-channel signal includes at least one of an applause sound characteristic and a general sound characteristic, based on the rendering type selecting at least one of a first downmix matrix and a second downmix matrix, and rendering a multichannel signal, wherein the rendering includes a rendering type such that the multichannel signal includes an applause sound characteristic. rendering the first frame using the first downmix matrix based on indicating and rendering the second frame using the second downmix matrix based on the rendering type indicating that the multichannel signal includes a general sound characteristic. rendering and providing a sound image having a sense of height with the plurality of output channel signals, wherein the layout of the plurality of output channel signals may be one of 5.0 channels or 5.1 channels.

Description

3D sound reproducing method and apparatus {Method and apparatus for 3D sound reproducing}

The present invention relates to a method and apparatus for reproducing a stereophonic sound, and more particularly to a method and apparatus for reproducing a multi-channel audio signal.

Thanks to the development of image and sound processing technology, high-definition, high-quality content is being produced in large quantities. Users who have requested high-definition and high-quality content want realistic images and sounds, and accordingly, research on stereoscopic images and stereophonic sound is being actively conducted.

Stereoscopic sound is a technology that allows a user to feel a sense of space by arranging a plurality of speakers at different positions on a horizontal plane and outputting the same or different sound signals from each speaker. However, actual sound can occur not only at various locations on the horizontal plane, but also at different altitudes. Accordingly, there is a need for a technique for reproducing sound signals generated at different altitudes through speakers arranged on a horizontal plane.

The present invention relates to a method and apparatus for reproducing a stereophonic sound, and to a method for reproducing a multi-channel audio signal including a high-level sound signal in a horizontal layout environment.

According to an embodiment of the present invention, a method for reproducing a stereophonic sound includes: acquiring a multi-channel audio signal; rendering a channel to be reproduced according to channel information and frequency of the multi-channel audio signal; and mixing the rendered signals.

The stereophonic sound reproducing method further includes the step of separating the applause signal from the multi-channel audio signal, wherein the rendering comprises either rendering the applause signal according to a 2D rendering method or each of the applause signal and rendering to the nearest channel among the output channels arranged on a horizontal plane for each channel.

The mixing may include mixing the rendered applause signal according to an energy boost method.

Separating the applause signal includes whether a non-tonal wideband signal exists in the multi-channel input signal, whether the level of the wideband signal is similar for each channel, whether an impulse shape of a short section is repeated, and correlation between channels determining whether the multi-channel input signal includes the applause signal based on at least one of whether ? and separating the applause signal according to the determination result.

The rendering may include dividing the multi-channel audio signal into a horizontal plane channel signal and an overhead channel signal based on the channel information; separating the overhead channel signal into a low frequency signal and a high frequency signal; rendering the low-frequency signal to the nearest channel among output channels arranged on a horizontal plane for each channel of the low-frequency signal; rendering the high-frequency signal according to a 3D rendering method; and rendering the horizontal channel signal according to a 2D rendering method.

The mixing may include: determining a gain to be applied to the rendered signals according to the channel information and the frequency; and applying and mixing the determined gain to the rendered signals.

The mixing may include mixing the rendered signals based on the power values of the rendered signals so that the power values are preserved.

A stereoscopic sound reproduction method according to an embodiment of the present invention includes: obtaining a multi-channel audio signal; rendering the multi-channel audio signal to a channel to be reproduced; and, based on the power value of the rendered signals, mixing the rendered signals so that the power value is preserved.

The mixing may include: mixing in units of a predetermined section based on the power values of the rendered signals; separating a low-frequency signal from among the rendered signals; The low frequency signal may include mixing the rendered signals based on the power values of the rendered signals in the previous section.

The rendering may include rendering to a channel to be reproduced according to channel information and frequency of the multi-channel audio signal.

A stereophonic sound reproducing apparatus according to an embodiment of the present invention includes: a renderer that obtains a multi-channel audio signal and renders it to a channel to be reproduced according to channel information and a frequency of the multi-channel audio signal; and a mixer for mixing the rendered signals.

A stereoscopic sound reproducing apparatus according to an embodiment of the present invention includes: a renderer that obtains a multi-channel audio signal and renders the multi-channel audio signal as a channel to be reproduced; and a mixer for mixing the rendered signals based on the power values of the rendered signals so that the power values are preserved.

According to an embodiment of the present invention, the stereophonic sound reproducing apparatus may reproduce the high-level component of the audio signal in a speaker disposed on a horizontal plane with a high-level sense.

According to an embodiment of the present invention, the stereophonic sound reproducing apparatus can minimize a change in tone or a disappearance of a sound when a multi-channel audio signal is reproduced in an environment with a small number of channels.

1 and 2 are block diagrams illustrating an internal structure of a stereoscopic sound reproducing apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a 3D sound reproduction method according to an embodiment of the present invention.
4 is a flowchart illustrating a stereoscopic sound reproduction method for an audio signal including an applause signal according to an embodiment of the present invention.
5 is a block diagram illustrating an internal structure of a 3D renderer according to an embodiment of the present invention.
6 is a flowchart illustrating a method of mixing a rendered audio signal according to an embodiment of the present invention.
7 is a flowchart illustrating a method of mixing a rendered audio signal according to a frequency according to an embodiment of the present invention.
8 is an exemplary diagram illustrating an example of mixing a rendered audio signal according to a frequency according to an embodiment of the present invention.
9 and 10 are block diagrams illustrating an internal structure of a stereoscopic sound reproducing apparatus according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, detailed descriptions of well-known functions or configurations that may obscure the gist of the present invention in the following description and accompanying drawings will be omitted. Also, it should be noted that, throughout the drawings, the same components are denoted by the same reference numerals as much as possible.

The terms or words used in the present specification and claims described below should not be construed as being limited to conventional or dictionary meanings, and the inventor shall appropriately define his/her invention as terms for the best description of his/her invention. Based on the principle that it can be done, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. Accordingly, the embodiments described in this specification and the configurations shown in the drawings are only the most preferred embodiment of the present invention, and do not represent all of the technical spirit of the present invention. It should be understood that there may be equivalents and variations.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

1 and 2 are block diagrams illustrating an internal structure of a stereoscopic sound reproducing apparatus according to an embodiment of the present invention.

The stereoscopic sound reproducing apparatus 100 according to an embodiment of the present invention may output a down-mixed multi-channel audio signal to a channel to be reproduced.

Stereophonic sound is a sound to which spatial information is added, which reproduces not only the pitch and tone of the sound, but also the direction and sense of distance to have a sense of presence, and to allow listeners who are not located in the space where the sound source is located to perceive the sense of direction, distance, and space. it means.

In the following description, a channel of an audio signal may mean the number of speakers from which sound is output. As the number of channels increases, the number of speakers outputting sound may increase. The stereoscopic sound reproducing apparatus 100 according to an embodiment of the present invention may render and mix a multi-channel audio signal with a channel to be reproduced so that a multi-channel audio signal with a large number of channels can be output and reproduced in an environment with a small number of channels. . In this case, the multi-channel audio signal may include a channel capable of outputting a high-level sound.

A channel capable of outputting a high-level sound may mean a channel capable of outputting an acoustic signal through a speaker located above a listener's head so as to feel a sense of elevation. The horizontal plane channel may mean a channel capable of outputting an acoustic signal through a speaker positioned on a horizontal plane with the listener.

The above-described environment in which the number of channels is small may mean an environment in which a sound can be output through a speaker disposed on a horizontal plane according to a horizontal plane channel without including a channel capable of outputting a high-level sound.

Also, in the following description, a horizontal channel may mean a channel including an audio signal that may be output through a speaker disposed on a horizontal plane. The overhead channel may refer to a channel including an audio signal that is disposed on an elevation rather than a horizontal plane and may be output through a speaker capable of outputting an elevation sound.

Referring to FIG. 1 , the stereoscopic sound reproducing apparatus 100 according to an embodiment of the present invention may include a renderer 110 and a mixer 120 .

According to an embodiment of the present invention, the stereophonic sound reproducing apparatus 100 may render, mix, and output a multi-channel audio signal as a channel to be reproduced. For example, the multi-channel audio signal may be a 22.2 channel signal, and the channel to be reproduced may be a 5.1 or 7.1 channel. The stereophonic sound reproducing apparatus 100 performs rendering by determining a channel to correspond to each channel of the multi-channel audio signal, and mixes the rendered audio signals by adding the signal of each channel corresponding to the channel to be reproduced and outputting it as a final signal. can

The renderer 110 may render a multi-channel audio signal according to a channel and a frequency. The renderer 110 may 3D (dimensionally) render and 2D (dimensionally) render a multi-channel audio signal according to an overhead channel and a horizontal plane channel, respectively.

The renderer 110 may render an overhead channel that has passed a head related transfer filter (HRTF) filter in 3D rendering methods according to frequencies in different ways. The HRTF filter not only shows simple path differences such as the level difference between the two ears and the time difference between the two ears, but also the complex path characteristics such as diffraction from the head surface and reflection from the pinna depending on the direction of sound arrival. It enables the recognition of stereophonic sound by changing phenomena. The HRTF filter may process the audio signals included in the overhead channel so that the stereophonic sound can be recognized by changing the sound quality of the audio signal.

The renderer 110 renders the low-frequency signal among the overhead channel signals according to the Add to the closest channel method, and the high-frequency signal is rendered according to the multichannel panning method. can do. According to the multi-channel panning method, a signal of each channel of the multi-channel audio signal may be respectively rendered to at least one horizontal plane channel by applying a gain value set differently for each channel to be rendered to each channel signal. Signals of each channel to which the gain value is applied may be combined through mixing and output as a final signal.

Since the low-frequency signal has strong diffraction properties, according to the multi-channel panning method, each channel of the multi-channel audio signal is not divided into several channels and rendered, but only one channel is rendered to have a similar sound quality for listeners to hear. Accordingly, the stereophonic sound reproducing apparatus 100 according to an embodiment of the present invention renders a low-frequency signal according to the add-to-closest-channel method, so that sound quality that can be generated when several channels are mixed into one output channel. deterioration can be prevented. That is, when several channels are mixed into one output channel, sound quality may be amplified or reduced and deteriorated according to interference between the respective channel signals, so that sound quality deterioration may be prevented by mixing one channel to one output channel.

According to the add-to-close channel method, each channel of the multi-channel audio signal may be rendered on the nearest channel among channels to be reproduced instead of being divided into multiple channels for rendering.

In addition, the 3D sound reproducing apparatus 100 may widen a sweet spot without degrading sound quality by performing rendering in a different method according to a frequency. That is, by rendering a low-frequency signal having strong diffraction characteristics according to the add-to-closest channel method, it is possible to prevent deterioration in sound quality that may occur when several channels are mixed into one output channel. The sweet spot refers to a predetermined range in which a listener can optimally listen to an undistorted stereophonic sound. As the sweet spot is wider, the listener can optimally listen to the undistorted stereophonic sound over a wide range, and when the listener is not located in the sweet spot, the listener can listen to the sound with distorted sound quality or sound image.

A method of rendering using a different panning method according to a frequency will be described in more detail below with reference to FIG. 4 or FIG. 5 .

The mixer 120 may combine the signals of the horizontal channels and the corresponding channels by the renderer 110 and output them as a final signal. The mixer 120 may mix signals of each channel for each predetermined section. For example, the mixer 120 may mix signals of each channel for each frame.

The mixer 120 according to an embodiment of the present invention may mix based on power values of signals rendered to each channel to be reproduced. In other words, the mixer 120 may determine the amplitude of the final signal or a gain to be applied to the final signal based on the power values of the signals rendered to the respective channels to be reproduced.

Referring to FIG. 2 , the stereoscopic sound reproducing apparatus 200 according to an embodiment of the present invention may include a sound analyzer 210 , a renderer 220 , a mixer 230 , and an output unit 240 . The 3D sound reproducing apparatus 200, the renderer 220, and the mixer 230 of FIG. 2 correspond to the 3D sound reproducing apparatus 100, the renderer 210, and the mixer 220 of FIG. 1 , and overlapping descriptions are omitted. decide to do

The sound analyzer 210 may analyze the multi-channel audio signal, select a rendering mode, and separate and output some signals included in the multi-channel audio signal. The acoustic analysis unit 210 may include a rendering mode selection unit 211 and a rendering signal separation unit 212 .

The rendering mode selector 211 may determine for each predetermined section whether there are many transient signals, such as a clap sound or a rain sound, in the multi-channel audio signal. In the following description, an audio signal having many transient, that is, an instantaneous and temporary signal, such as an applause sound or a rain sound, will be referred to as an applause signal.

The stereoscopic sound reproducing apparatus 200 according to an embodiment of the present invention may separate the applause signal and process channel rendering and mixing according to the characteristics of the applause signal.

The rendering mode selector 211 may select the rendering mode as either a general mode or an applause mode according to whether the applause signal is included in the multi-channel audio signal. The renderer 220 may render according to the mode selected by the rendering mode selector 211 . That is, the renderer 220 may render the applause signal according to the selected mode.

The rendering mode selector 211 may select a normal mode when the applause signal is not included in the multi-channel audio signal. According to the normal mode, the overhead channel signal may be rendered by the 3D renderer 221 , and the horizontal channel signal may be rendered by the 2D renderer 222 . That is, rendering may be performed without considering the applause signal.

The rendering mode selection unit 211 may select the applause mode when the applause signal is included in the multi-channel audio signal. According to the applause mode, the applause signal may be separated, and rendering may be performed on the separated applause signal.

The rendering mode selection unit 211 determines whether the applause signal is included in the multi-channel audio signal for each predetermined section using applause bit information included in the multi-channel audio signal or separately received from another device. can Applause bit information includes bsTsEnable or bsTempShapeEnableChannel flag information according to the MPEG-based codec, and a rendering mode may be selected by the rendering mode selector 211 according to the above-described flag information.

Also, the rendering mode selector 211 may select a rendering mode based on the characteristics of the multi-channel audio signal of a predetermined section to be determined. That is, the rendering mode selector 211 may select the rendering mode according to whether the characteristic of the multi-channel audio signal of a predetermined section has the characteristic of the audio signal including the applause signal.

The rendering mode selector 211 determines whether a non-tonal wideband signal exists in a plurality of input channels in a multi-channel audio signal of a predetermined section, and whether the level of the signal is similar for each channel, and determines whether the signal level is similar for each channel. It may be determined whether the applause signal is included in the multi-channel audio signal based on at least one condition of whether the impulse form is repeated and whether the correlation between channels is low. .

The rendering mode selector 211 may select the rendering mode as the applause mode when it is determined that the applause signal is included in the multi-channel audio signal in the current section.

The rendering signal separation unit 212 may separate the applause signal included in the multi-channel audio signal from the general sound signal when the applause mode is selected by the rendering signal selection unit 211 .

When the bsTsdEnable flag in the MPEG USAC series is used, 2D rendering may be performed like a horizontal channel signal according to flag information regardless of the elevation of the corresponding channel. In addition, the overhead signal may also be mixed by being assumed to be a horizontal channel signal according to flag information. That is, the rendering signal separating unit 212 may separate the applause signal included in the multi-channel audio signal of a predetermined section according to the flag information, and the separated applause signal may be 2D rendered like the horizontal channel signal. there is.

When the flag is not used, the rendering signal separation unit 212 may analyze a signal between channels to separate the applause signal component. The separated applause signal among the overhead signals may be 2D rendered, and the remaining signals other than the applause signal may be 3D rendered.

The renderer 220 may include a 3D renderer 221 that renders an overhead signal according to a 3D rendering method and a 2D renderer 222 that renders a horizontal channel signal or applause signal according to a 2D rendering method.

The 3D renderer 221 may render the overhead signal in different ways according to frequencies. The 3D renderer 221 may render a low-frequency signal according to the add-to-close channel method, and render a high-frequency signal according to the 3D rendering method. Hereinafter, the 3D rendering method refers to a method of rendering an overhead signal, and the 3D rendering method may include a multi-channel panning method.

The 2D renderer 222 may render the horizontal channel signal or the applause signal according to at least one of a 2D rendering method, an add-to-closest channel method, and an energy boost method. Hereinafter, the 2D rendering method refers to a method of rendering a horizontal channel signal, and the 2D rendering method may include a downmix equation or a VBAP method.

The 3D renderer 221 and the 2D renderer 222 may be simplified by matrixing, respectively. The 3D renderer 221 may perform downmixing through a 3D downmix matrix that may be determined as a function of an input channel, an output channel, and a frequency. The 2D renderer 222 may be downmixed through a 2D downmix matrix that may be determined as a function of an input channel and an output channel. That is, the 3D or 2D downmix matrix includes coefficients that can be determined according to an input channel, an output channel, or a frequency, so that the input multi-channel audio signal can be downmixed.

Since the amplitude part for each frequency is more important than the phase part of the sound signal during rendering, the 3D renderer 221 and the 2D renderer 222 render using a downmix matrix including coefficients that can be determined according to frequency values, respectively. By doing so, the amount of computation of rendering can be simplified. The signal rendered through the downmix matrix may be mixed according to the power conservation module of the mixer 230 and output as a final signal.

The mixer 230 may calculate the rendered signals for each channel and output a final signal. The mixer 230 according to an embodiment of the present invention may mix rendered signals based on power values of signals included in each channel. Accordingly, the stereophonic sound reproducing apparatus 200 according to an embodiment of the present invention can reduce tone distortion that may occur due to frequency reinforcement or cancellation by mixing based on the power value of the rendered signal.

The output unit 240 may finally output the signal mixed by the mixer 230 through the speaker. In this case, the output unit 240 may output the sound signal through different speakers according to the channel of the mixed signal.

3 is a flowchart illustrating a 3D sound reproduction method according to an embodiment of the present invention.

Referring to FIG. 3 , in step S301 , the stereophonic sound reproducing apparatus 100 may render a multi-channel audio signal according to channel information and frequency. The stereophonic sound reproducing apparatus 100 may perform 3D rendering or 2D rendering according to channel information, but may render a low-frequency signal in consideration of characteristics of the low-frequency signal.

In operation S303, the stereophonic sound reproducing apparatus 100 may generate a final signal by mixing the signals rendered in operation S301. The stereophonic sound reproducing apparatus 100 may render by determining a channel through which signals of each channel of the multi-channel audio signal are to be output, mix by summing or calculating the rendered signals, and may generate a final signal.

4 is a flowchart illustrating a stereoscopic sound reproduction method for an audio signal including an applause signal according to an embodiment of the present invention.

Referring to FIG. 4 , in step S401 , the stereophonic sound reproducing apparatus 200 may analyze the multi-channel audio signal for each predetermined section to determine whether the applause signal is included in the multi-channel audio signal.

In step S403, the stereophonic sound reproducing apparatus 200 may determine whether the applause signal is included in the input multi-channel audio signal for each predetermined section. For example, it may be determined for each frame. The stereophonic sound reproducing apparatus 200 may determine whether the applause signal is included for each predetermined section by analyzing flag information or a multi-channel audio signal of a predetermined section to be determined. The stereophonic sound reproducing apparatus 200 separates and processes the applause signal from the overhead signal or the horizontal channel signal, thereby minimizing sound quality distortion that may occur when mixing the applause signal.

If it is determined in step S405 that the applause signal is included, the stereophonic sound reproducing apparatus 200 separates the applause signal and 2D renders the applause signal and the horizontal channel signal in step S407. there is.

A signal of a horizontal plane channel may be 2D rendered according to a downmix equation or a vector base amplitude panning (VBAP) method.

The applause signal may be rendered to the nearest channel when a channel including an elevation sound is projected on a horizontal plane according to the add-to-close channel method, or may be rendered according to the 2D rendering method and then mixed according to the energy boost method.

When the applause signal is rendered and mixed according to the 2D or 3D rendering method, the number of transient components in the mixed signal increases and whitening occurs or the cross-correlation between channels increases and the sound image deteriorates. can be narrowed. Therefore, in order to prevent the whitening phenomenon or the narrowing of the sound image, the stereophonic sound reproducing apparatus 200 renders the applause signal according to the add-to-closest channel method or the energy boost method used for 3D rendering of the low-frequency signal, can be mixed

The energy boost method refers to a method of mixing by increasing the energy of a horizontal channel signal in order to prevent tone whitening due to a change in a transient period when audio signals of multiple channels are mixed into one channel. The energy boost method relates to a method of mixing a rendered applause signal.

A method of mixing the applause signal according to the energy boost method may be performed according to Equation 1 below.

[Equation 1]

(주파수 도메인에서의 처리)

(Processing in the frequency domain)

ω _{in, out} means downmixing gain, and the applause signal is rendered as a channel to which each channel of a multi-channel audio signal is to be reproduced, and when mixed, a downmixing gain may be applied to each channel. . The downmixing gain may be predetermined as a predetermined value according to a channel to which each channel is rendered. x _in=out [l,k] represents an applause signal rendered to correspond to the output layout, and means any applause signal. l is a value for identifying a predetermined section of an acoustic signal, and k represents a frequency. x _in=out [l,k]/|x _in=out [l,k]| represents the phase value of the input applause signal, and the values in the root of Equation 1 may mean the sum of the power, ie, energy values, of applause signals corresponding to the same output channel.

Referring to Equation 1, the gain of each channel to be reproduced may be modified by the power value of values to which the down-mixing gain is applied to a plurality of applause signals rendered to one channel of the output layout. Accordingly, the amplitude of the applause signal may be increased by the sum of the energy values, and a whitening phenomenon that may occur due to a phase difference may be prevented.

In operation S409 , when it is determined that the applause signal is not included, the stereophonic sound reproducing apparatus 200 may 2D render the signal of the horizontal channel.

In operation S411, the stereophonic sound reproducing apparatus 200 may filter the overhead channel signal with an HRTF filter to provide the stereophonic signal. When the overhead channel signal is a signal of a frequency domain or a filter bank sample, HRTF filtering can be performed by simple multiplication because it is a filter for providing only relative weighting of a spectrum.

In operation S413 , the stereophonic sound reproducing apparatus 200 may separate the overhead channel signal into a high frequency and a low frequency. For example, the 3D sound reproducing apparatus 200 may separate a sound signal having a frequency less than or equal to 1 kHz into a low frequency. The low-frequency component may be rendered according to an add-to-closest channel method according to an acoustic characteristic with strong diffraction.

In operation S415, the stereophonic sound reproducing apparatus 200 may render a signal separated into a high frequency signal according to a 3D rendering method. The 3D rendering method may include a multi-channel panning method. Multi-channel panning may mean that each channel signal of a multi-channel audio signal is distributed to channels to be reproduced. In this case, each channel signal to which the panning coefficient is applied may be distributed to channels to be reproduced. In the case of a high-frequency signal, the signal may be distributed to a surround channel in order to provide a characteristic that an interaural level difference (ILD) between the two ears decreases as the sense of height increases. Also, the direction of the acoustic signal may be determined by the number of the front channel and the plurality of channels to be panned.

In operation S417, the stereophonic sound reproducing apparatus 100 may render the low-frequency signal according to the above-described add-to-closest channel method. When many signals in one channel, that is, several channel signals of a multi-channel audio signal are mixed, sound quality may be deteriorated as sound quality is canceled or amplified due to different phases. According to the add-to-close channel method, the stereophonic sound reproducing apparatus 100 may map each channel to the closest channel when projected on the horizontal plane as shown in Table 1 below in order to prevent the above-described sound quality deterioration.

Input Channels (22.2) Output channels (5.1) Top Front Left(TFL) Front Left(FL) Top Front Right (TFR) Front Right (FR) Top Surr Left (TSL) Surround Left (SL) Top Surr Right (TSR) Surround Right (SR) Top Back Left (TBL) Surround Left (SL) Top Back Right (TBR) Surround Right (SR) Top Front Center (TFC) Front Center (FC) Top Back Center (TBC) Surrounds (SL & SR) Voice of God (VOG) Front & Surr (FL, FR, SL, SR)

Referring to Table 1, channels such as TBC and VOG having a plurality of close channels among the overhead channels may be allocated to 5.1 channels by a panning coefficient for sound image localization. Mapping shown in Table 1 The relationship is merely an example, and the respective channels may be mapped differently without being limited thereto.

When the multi-channel audio signal is a frequency signal or filter bank signal, the bin or band corresponding to the low frequency is the add-to-close channel method, and the bin or band corresponding to the high frequency is the multi-channel method. It may be rendered according to a channel panning method. A bin or a band may mean a signal period of a predetermined unit in the frequency domain.

In operation S419, the stereophonic sound reproducing apparatus 100 may mix signals rendered in each channel based on a power value. In this case, the stereophonic sound reproducing apparatus 100 may perform mixing in the frequency domain. A method of mixing signals rendered to each channel based on a power value will be described in more detail below with reference to FIGS. 6 to 7 .

In operation S421, the stereophonic sound reproducing apparatus 100 may output a mixed final signal.

5 is a block diagram illustrating an internal structure of a 3D renderer according to an embodiment of the present invention. The 3D renderer 500 of FIG. 5 corresponds to the 3D renderer 221 of FIG. 2 , and overlapping descriptions will be omitted.

Referring to FIG. 5 , the 3D renderer 500 may include an HRTF filter 510 , an LPF 520 , an HPF 530 , an add-to-closest channel 540 , and a multi-channel panning 550 . .

The HRTF filter 510 may HRTF filter an overhead channel signal among multi-channel audio signals.

The LPF 520 may separate and output a low-frequency component of the HRTF-filtered overhead channel signal.

The HPF 320 may separate and output a high-frequency component of the HRTF-filtered overhead channel signal.

The add-to-close channel 540 may render a channel closest to the low-frequency component of the overhead channel signal when it is projected onto the horizontal plane of each channel.

The multi-channel panning 550 may render a high-frequency component of the overhead channel signal according to the multi-channel panning method.

6 is a flowchart illustrating a method of mixing a rendered audio signal according to an embodiment of the present invention. Steps S601 to S605 of FIG. 6 correspond to steps S419 of FIG. 4 , and overlapping descriptions will be omitted.

Referring to FIG. 6 , in operation S601 , the stereophonic sound reproducing apparatus 100 may obtain a rendered audio signal.

In operation S603 , the stereophonic sound reproducing apparatus 100 obtains power values of signals rendered for each channel, and mixes them based on the power values obtained for each channel in operation S605 to generate a final signal.

7 is a flowchart illustrating a method of mixing a rendered audio signal according to a frequency according to an embodiment of the present invention. Steps S701 and S703 of FIG. 7 correspond to steps S601 and S603 of FIG. 6 , and overlapping descriptions will be omitted.

Referring to FIG. 7 , in step S701 , the stereophonic sound reproducing apparatus 100 may obtain a rendered audio signal.

In step S703, the stereophonic sound reproducing apparatus 100 may obtain power values of the rendered signals for each channel according to a power preserving module, and in step S705, mix them based on the obtained power values . Power values of signals rendered for each channel may be obtained by calculating the sum of squares of signals rendered for each channel.

[Equation 2]

(주파수 도메인에서의 처리)

(Processing in the frequency domain)

x _{in and out} are audio signals rendered on any one channel, x _out indicates the total sum of signals rendered on any one channel, and l indicates a current section of the multi-channel audio signal. k represents the frequency, and y _out represents the signal mixed according to the power conservation module.

According to the power conservation module, the power of the final mixed signal may be mixed based on the power values of the signals rendered for each channel described above so that the power before mixing may be maintained. Accordingly, according to the power conservation module, it is possible to prevent the acoustic signal from being distorted due to constructive or destructive interference when the mixed signal is added to the rendered signals.

Referring to Equation 2, the stereophonic sound reproducing apparatus 100 applies the power values of the rendered signals for each channel to the phase of the total sum of the signals rendered for each channel according to the power conservation module. You can mix signals.

When the signal obtained in step S701 is in the time domain, it may be converted to the frequency domain and then mixed according to Equation (2). In this case, the acoustic signal in the time domain may be converted into the frequency domain according to a frequency or a filter bank scheme.

However, when the stereophonic sound reproducing apparatus 100 applies the power conservation module to each predetermined section, the power value of each signal is estimated for each predetermined section. not enough compared to Accordingly, since the estimated power value may vary for each section, a discontinuous portion may occur at the boundary of the section to which the power conservation module is applied. On the other hand, in the case of a high frequency, since an interval for estimating the power value is sufficient compared to the wavelength, the possibility that a discontinuous portion is generated at the boundary of the interval may be low. That is, a one-pole smoothing method, which will be described below, may be applied depending on whether an interval for estimating a power value is sufficient compared to a wavelength.

The stereophonic sound reproducing apparatus 100 according to an embodiment of the present invention may determine, in step S707, whether a portion corresponding to the low-frequency signal exists in the signal mixed in step S705. When a part corresponding to the low-frequency signal exists in the mixed signal, the stereophonic sound reproducing apparatus 100 generates at the boundary of the section to which the power conservation module is applied using the one-pole smoothing method of Equation 3 below in steps S709 to S711 discontinuous parts can be removed.

[Equation 3]

(주파수 도메인에서의 처리)

(Processing in the frequency domain)

P _out may be obtained based on the power value of the total sum of P _out of the previous section and the mixed signal of the current section.

P _in may be obtained based on the total sum of P _in of the previous section and the power values of each rendered signal of the current section.

The power value of the previous section may be applied to the equation according to γ that can be applied to P _out or P _in of the previous section, and γ may be determined to have a smaller value as the wavelength of the low frequency is longer or the frequency value is smaller.

The stereophonic sound reproducing apparatus 100 according to an embodiment of the present invention adjusts the gain of the mixed signal based on the power value of the signals rendered in the previous section or the signals obtained by adding the rendered signals in order to remove the discontinuous portion. can be adjusted

In addition, similarly to Equation 3, the gain of the output signal may be obtained based on the gain value of the output signal of the previous section, and thus may be processed as in Equation 4 to remove the discontinuous portion.

[Equation 4]

(주파수 도메인에서의 처리)

(Processing in the frequency domain)

The stereophonic sound reproducing apparatus 100 according to an embodiment of the present invention removes a discontinuous portion, and a gain of a mixed signal based on a gain applied to signals rendered in a previous section or a signal to which rendered signals are added. can be adjusted.

8 is an exemplary diagram illustrating an example of mixing a rendered audio signal according to a frequency according to an embodiment of the present invention.

Referring to FIG. 8 , referring to the signal 803 during the mixing process by adding the rendered audio signals 801 and 802, the amplitude of the added value of the rendered audio signals 801 and 802 is amplified due to the phase difference. As it is, the sound may become louder.

Accordingly, the stereophonic sound reproducing apparatus 100 according to an embodiment of the present invention applies the power conservation module to determine the gain of the signal 803 during the mixing process based on the power values of the rendered audio signals 801 and 802 . can be decided by

The signal 804 mixed according to the power conservation module was adjusted to have an amplitude similar to that of the rendered audio signals 801 and 802, but as the power conservation module is applied for each predetermined section, a discontinuous portion is included in each section can do.

Accordingly, the stereophonic sound reproducing apparatus 100 according to an embodiment of the present invention may obtain the final signal 805 by smoothing the mixed signal with reference to the power value of the previous section according to the one-pole smoothing technique.

9 and 10 are block diagrams illustrating an internal structure of a stereoscopic sound reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 9 , the 3D sound reproducing apparatus 900 may include a 3D renderer 910 , a 2D renderer 920 , a weighting application unit 930 , and a mixer 940 . The 3D renderer 910, the 2D renderer 920, and the mixer 940 of FIG. 9 correspond to the 3D renderer 221, the 2D renderer 222, and the mixer 230 of FIG. 2, and overlapping descriptions may be omitted. there is.

The 3D renderer 910 may render an overhead channel signal among multi-channel audio signals.

The 2D renderer 920 may render a horizontal plane channel signal among multi-channel audio signals.

The weighting application unit 930 is a component for outputting a multi-channel audio signal according to the channel layout to be reproduced when the channel layout of the signal to be reproduced does not match the channel layout of the signal to be reproduced among the layouts that can be rendered by the 3D renderer 910 . The layout of the channel to be reproduced may mean arrangement information of speakers to which the channel signal to be reproduced will be output.

When the 2D renderer 920 renders using the VBAP method, it is possible to render a horizontal channel signal even in an arbitrary layout channel environment. According to the VBAP method, the stereophonic sound reproducing apparatus 900 can render a multi-channel audio signal by obtaining a panning gain in an arbitrary speaker environment only by simple vector-based calculation. Accordingly, the weight may be determined according to the degree to which any reproduction channel layout is similar to the layout rendered by the 3D renderer 910 . For example, when the 3D renderer 910 renders a multi-channel audio signal in a 5.1-channel playback environment, the weighting can be determined depending on how different the layout of an arbitrary layout channel environment to be rendered is from the 5.1-channel playback environment. there is.

In addition, the 3D weighting application unit 930 may apply and output the determined weighting to the signals rendered by the 3D renderer 910 and the 2D renderer 920 , respectively.

Referring to FIG. 10 , the stereoscopic sound reproducing apparatus 1000 may include a 3D renderer 1010 , a 2D renderer 1020 , and a mixer 1030 . The 3D renderer 1010 , the 2D renderer 1020 , and the mixer 1030 of FIG. 9 correspond to the 3D renderer 221 , the 2D renderer 222 , and the mixer 230 of FIG. 2 , and overlapping descriptions may be omitted. there is.

The 3D renderer 1010 may render a layout most similar to a layout of a channel to be output among renderable layouts. In addition, the 2D renderer 1020 may render the signal rendered by the 3D renderer 1010 by re-panning to the channel layout of the signal to be output for each channel.

For example, when the 3D renderer 1010 renders a multi-channel audio signal in a 5.1-channel playback environment, the 2D renderer 1020 renders a 3D-rendered signal according to an arbitrary layout channel environment to be rendered according to the VBAP method. It can be rendered by panning again.

According to an embodiment of the present invention, the stereophonic sound reproducing apparatus may reproduce the high-level component of the audio signal in a speaker disposed on a horizontal plane with a high-level sense.

According to an embodiment of the present invention, the stereophonic sound reproducing apparatus can minimize a change in tone or a disappearance of a sound when a multi-channel audio signal is reproduced in an environment with a small number of channels.

The method according to an embodiment of the present invention can be implemented as computer-readable code on a computer-readable recording medium (including all devices having an information processing function). The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording devices include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device.

Although the foregoing description has focused on novel features of the invention as applied to various embodiments, those skilled in the art will recognize the apparatus and method described above without departing from the scope of the invention. It will be understood that various deletions, substitutions, and changes are possible in the form and details of Accordingly, the scope of the invention is defined by the appended claims rather than by the foregoing description. All modifications within the scope of equivalents of the claims are included in the scope of the present invention.

Claims (10)

receiving a multi-channel signal comprising a height input channel signal and a rendering type, wherein the multi-channel signal includes at least one of an applause sound characteristic and a general sound characteristic;
selecting at least one of a first downmix matrix and a second downmix matrix based on the rendering type; and
rendering the multi-channel signal;
The rendering step
rendering a first frame using the first downmix matrix based on the rendering type indicating that the multi-channel signal includes the applause sound characteristic; and
Rendering a second frame using the second downmix matrix based on the rendering type indicating that the multi-channel signal includes the general sound characteristic, and providing a sound image having a sense of height with a plurality of output channel signals carried out by at least one of the steps of
The layout of the plurality of output channel signals is one of 5.0 channels or 5.1 channels. The method according to claim 1, wherein the rendering type is identified from a parameter included in a bitstream. The method of claim 1 , wherein the rendering type is identified based on at least one of a bandwidth of the multi-channel signal and a correlation between channels of the multi-channel signal. The method of claim 1, wherein the rendering comprises:
and if the multi-channel signal includes the applause sound characteristic, rendering the first frame by two-dimensional rendering. The method of claim 1, wherein the rendering comprises:
based on the power value of the multi-channel signal, rendering the multi-channel signal so that the power value is preserved. a receiver configured to receive a multi-channel signal including a height input channel signal and a rendering type, wherein the multi-channel signal includes at least one of an applause sound characteristic and a general sound characteristic; and
contains the renderer,
the renderer selects at least one of a first downmix matrix and a second downmix matrix based on the rendering type;
rendering a first frame by using the first downmix matrix based on the rendering type indicating that the multi-channel signal includes the applause sound characteristic, and the rendering type indicating that the multi-channel signal includes the rendering a second frame using the second downmix matrix on the basis of indicating including general sound characteristics to provide a sound image having a sense of height as a plurality of output channel signals;
The layout of the plurality of output channel signals is one of 5.0 channels or 5.1 channels. The audio signal rendering apparatus according to claim 6, wherein the rendering type is identified from a parameter included in a bitstream. The apparatus of claim 6 , wherein the rendering type is identified based on at least one of a bandwidth of the multi-channel signal and a correlation between channels of the multi-channel signal. The audio signal rendering apparatus according to claim 6, wherein the renderer renders the first frame by two-dimensional rendering when the multi-channel signal includes the applause sound characteristic. The audio signal rendering apparatus according to claim 6, wherein the renderer renders the multi-channel signal so that the power value is preserved based on the power value of the multi-channel signal.

Images