KR100673288B1 - System for providing audio data and providing method thereof - Google Patents

Abstract

The present invention relates to an audio data providing system and a method for providing audio data. An audio data providing system according to the present invention is connected to a user terminal through a network and provides audio data to the user terminal. The system includes a condition setting unit and an audio data processing unit.

When the audio data providing condition including the virtual sound source position condition is set by the user in the condition setting unit, the audio data processing unit convolves the input audio signal and a predetermined transfer function according to the set virtual sound source position condition, thereby allowing the user to input the audio signal. Outputs by giving a three-dimensional effect in the desired direction.

Therefore, the user can enjoy the sound given the three-dimensional effect in a desired direction even through the network.

HRTF, Cross Talk, Downmixing, Head Transfer Function

Description

Audio data providing system and method for providing audio data {SYSTEM FOR PROVIDING AUDIO DATA AND PROVIDING METHOD THEREOF}

1 is an overall configuration diagram of an audio data providing system according to an embodiment of the present invention.

FIG. 2 is a detailed configuration diagram of the service providing server of FIG. 1.

3 is a diagram illustrating a relationship between an input signal and a signal heard by a user's ear.

4 is an overall flowchart of an audio providing method according to an exemplary embodiment of the present invention.

5 is a two-channel downmix process when the number of channels of the input audio signal is three or more.

6 is a process of downmixing two channels after expanding the number of channels when the number of channels of the input audio signal is 2 or less.

FIG. 7 is a process of finally outputting audio data that has undergone the process of FIG. 5 or 6.

 * Explanation of symbols for the main parts of the drawing *

100: user terminal 200: network

300: interface 400: service providing server

410: audio source providing unit 420: channel identification unit

430: upmix processing unit 433: channel expansion unit

440: downmix processing unit 450: crosstalk processing unit

460: condition setting unit 470: output unit

The present invention relates to an audio data providing system and an audio data providing method, and more particularly, to an audio data providing system connected to a user terminal through a network and providing audio data to the user terminal.

The audio data provided to the user uses 5.1 channels to enhance the stereoscopic sound. In this case, the 5.1 channel refers to left, right, center, surround left, surround right and low frequency effect channels.

In general, the center channel has a direction component at the center of the listener, the left and right channels at the left and right 30 degree positions, and the left and right surround channel at the left and right 120 degrees positions, and the low frequency effect channel has a bandwidth of 120 Hz or less and thus has no direction component.

Here, in order to hear a three-dimensional sound, the speaker position corresponding to five channels except the low frequency effect channel having no directionality is important. That is, three-dimensional speaker sound is formed by the five speaker positions.

On the other hand, the recent method of providing and reproducing video or audio data through a network is mainly performed by a method of reproducing in real time without downloading, that is, a streaming method, rather than a method of downloading and reproducing conventional data to a hard disk drive.

However, in the case of providing the audio data by the streaming method, it is difficult to give a three-dimensional sound of the sound through the five-channel audio data transmission and the speaker position corresponding to the five channels. If 5-channel audio data is transmitted through the network as it is, it may be impossible to transmit in real time because a large amount of data must be transmitted in a limited bandwidth. Therefore, a method of down-mixing 5-channel audio data into two-channel audio data of left and right has been used.

However, when downmixing each of the five channel audio data into left and right two channel audio data by simply dividing each of the left and right components, it is difficult to maintain a three-dimensional sound. Therefore, even when downmixing 5-channel audio data into 2 channels, a method of performing bi- ural synthesis on each of the 5-channel audio data has been used to maintain sound stereoscopic effect.

The pair effect refers to the effect of detecting a sound's position, that is, the sound source's direction and perspective by listening to one ear when listening to a sound in a person's hearing. Basic principle. Therefore, the paired effect synthesis means that a non-directional mono channel is passed through a Head Related Trasnfer Function (HRTF) filter having a specific angle and converted into a signal having a directivity.

In other words, by giving direction to each of the five-channel audio data, downmixing and streaming to two channels, it was possible to stream the service to maintain the three-dimensional sound while reducing the amount of data.

However, the five sound source positions officially recommended for listening to a three-dimensional sound are the center of the listener (i.e. 0 degrees from the listener), the left and right channels are at the left and right 30 degrees positions, and the left and right surround channels are at 120 degrees. The two-channel downmixing method followed by the above recommendations. That is, although various virtual sound source positions can be set using the head transfer function, five virtual sound source positions are fixed to the center of the listener, 30 degrees to the left and right of the listener, and 120 degrees to the listener's left and right, regardless of the user's individual preference.

On the other hand, when listening to the audio data down to two channels through the headphone, the left channel into the left ear, the right channel into the right ear, the canceling effect due to signal interference between the channels (hereinafter referred to as cross-talk) Does not occur. However, when listening to 2-channel audio data through a speaker, crosstalk occurs because a part of the left channel enters the right ear and a part of the right channel enters the left ear. Therefore, a process for removing the crosstalk is necessary separately.

However, in general, in the audio data streaming service through a network, there is no consideration of crosstalk or a crosstalk removal process is added regardless of whether a sound output means is a speaker or a headphone.

In addition, the crosstalk removal process is performed uniformly through a presumed function (typically 30 degrees left and right from the user) regardless of the speaker actual setting position from the user.

Therefore, the technical problem to be achieved by the present invention is to provide audio data with a three-dimensional effect in a direction desired by an audio data streaming service.

In addition, another technical problem to be achieved by the present invention is to determine whether or not crosstalk processing according to the provided audio data output means conditions.

In addition, another technical problem to be achieved by the present invention is to perform a crosstalk process based on the user's speaker setting position.

An audio providing system according to a feature of the present invention for achieving the above technical problem, is an audio data providing system for providing audio data to the user terminal is connected to the user terminal via a network, the user specified virtual sound source position conditions A condition setting unit for setting audio data providing conditions including; And an audio data processor for providing a three-dimensional effect in a direction desired by a user using a predetermined transfer function according to the virtual sound source position condition set in the condition setting unit.

At this time, the condition setting unit may be configured to change the condition setting in real time by the user.

The apparatus may further include a crosstalk processing unit for canceling an offset effect due to mutual interference of the output audio signal, wherein the crosstalk processing unit is configured to be executed according to an output means condition set in the condition setting unit.

In particular, the output means condition may include a kind of output means such as a headphone or a speaker, and the crosstalk processing unit may be configured to execute an audio signal canceling effect canceling function when the output means condition is a speaker. At this time, the output means condition further includes a speaker position from the user, and the crosstalk processing unit is configurable to execute an audio signal canceling effect canceling function based on the set speaker position.

The audio data providing method of the audio data providing system according to an aspect of the present invention relates to an audio data providing method of a system that is connected to a user terminal through a network and provides the audio data to the user terminal. Setting audio data providing conditions including virtual sound source position conditions; b) providing a three-dimensional effect in a direction desired by a user using a predetermined transfer function according to the set audio data providing condition; c) outputting the audio signal.

In this case, a) may be changed in real time by the user. And d) selecting a kind of output means; And e) when the output means is a speaker, removing the offset effect due to mutual interference of the audio signal.

If the output means is a speaker, the method may further include setting a speaker position from the user, thereby eliminating an audio signal canceling effect based on the set speaker position.

In addition, an audio providing system according to another aspect of the present invention is an audio data providing system for providing audio data to a user terminal connected to a user terminal through a network, the output means including a type of output means of audio data designated by a user A condition setting unit for setting a condition; And a crosstalk processing unit for canceling an offset effect due to mutual interference of the output audio signal, wherein the crosstalk processing unit is configurable to be executed according to the output means condition set in the condition setting unit.

The audio data providing method according to another aspect of the present invention is an audio data providing method of a system that is connected to the user terminal through a network and provides the audio data to the user terminal. Setting an output means condition including; b) if the type of output means is a speaker, comprising the step of eliminating the cancellation effect due to mutual interference of the audio signal.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIG. 1, an audio data providing system according to an exemplary embodiment of the present invention includes a plurality of user terminals through a network 200 (including all types of networks 200 in various forms such as a telephone network, the Internet, and a wireless communication network). It is connected to the 100, and includes an interface 300 and the service providing server 400.

The user terminal 100 is a communication device that can be connected to the system through the network 200, and generally refers to various communication devices such as wired telephones, wireless communication terminals, computers, and Internet-accessible TVs. Means a communication device that can receive and output data.

The interface 300 includes a database interlocking device (CGI) for transmitting and receiving information with a web server or other system, and the plurality of user terminals 100 through a network 200, in particular, a wired or wireless Internet. To allow them to connect. In addition, various information received from the service providing server 400 executing the audio data providing service is converted into a communication standard and provided to the plurality of user terminals 100, and the user terminal 100 is provided through the network 200. Receives the transmitted information and delivers it to the service providing server (400).

The service providing server 400 provides an audio data providing service to the user terminal 100 and provides an audio source providing unit 410, an audio data processing unit, a crosstalk processing unit 450, and a condition setting unit 460. ) And an output unit 470. The audio data processor includes a channel checker 420, an upmix processor 430, and a downmix processor 440.

The output unit 470 outputs the final result of the audio data processed by the audio data providing system to be delivered to the user terminal 100.

Unlike the headphones, the crosstalk processing unit 450 is a crosstalk phenomenon caused by a part of the left channel entering the right ear and a part of the right channel entering the left ear when listening to 2-channel audio data through a speaker. Remove

The condition setting unit 460 sets the audio data providing condition specified by the user. The audio data provision condition includes the virtual sound source position, the type of output means, and the speaker position from the user. In addition, the condition setting is configurable to be changed in real time by the user, there is an advantage that the user can immediately respond to the desired condition.

Here, the virtual sound source position designation designates a direction in which the user wants to listen to each of the five channel audio data. For example, the center, left, and right surround channels can be set to 0 degrees, 30 degrees left and right, 120 degrees left and right positions from the user center, and follow the standard recommendations, and various other settings are possible.

For example, the type of output means means that the user can listen to the audio data transmitted through the network 200 through the speaker or headphones (hereinafter, including all sound output devices directly delivered to the human ear regardless of the actual name). By setting whether or not to listen through, it can be configured to enable a variety of other output means type setting. At this time, whether to execute the crosstalk processing unit 450 is determined according to the type of output means.

The speaker position from the user indicates an angle of the speaker away from the user with respect to the user, and the crosstalk removal function of the crosstalk processing unit 450 varies according to this position.

The audio data processing unit provides a two-dimensional downmix process to the sound source received from the audio source providing unit 410.

As shown in FIG. 2, the audio data processing unit includes a channel identification unit 420 and an upmix processing unit 430 which expands the number of channels when the number of channels of the sound source is 2 or less and gives a three-dimensional effect to sound by downmixing the two channels. And a down mixer processing unit for providing a three-dimensional effect to the sound and downmixing the two channels when the number of channels of the sound source is three or more.

At this time, the channel checking unit 420 checks the number of channels of the sound source provided from the audio source, and transfers the sound source to the upmix processor 430 or the downmix processor 440 according to the number.

The upmix processor 430 processes audio data when the number of channels of a sound source is 2 or less, and includes a decoder, a channel expander 433, a pair effect synthesizer, a downmixer, and an encoder.

The decoder divides the received sound source into left and right channels.

The channel expansion unit 433 expands 2-channel audio data into 4-channel audio data. The channel expansion unit 433 bypasses the left and right channels received from the decoder, and multiplies an appropriate gain value with the left and right channel signals as described below, and generates the surround left and right signals by adding the multiplied values.

Left Surround = (gain1 * left) + (gain2 * right)

Right Surround = (gain2 * left) + (gain1 * right)

That is, the channel expansion unit 433 extends the two channel audio data of the left and right channels into four channel audio data of the left and right and surround left and right channels.

Next, the pair effect synthesizing unit gives directionality to each channel through pair effect synthesizing based on a head transfer function, and separates four channel audio data into eight signal components. The head transfer function corresponding to the virtual sound source position condition set in is used. In general, the head transfer function corresponding to ± 60 degrees for the left / right signal and ± 120 degrees for the surround left / right signal is used.

The downmixer combines each of the eight signal components into left and right signal components, adds them to an appropriate size, and makes two channels. The two-channel audio data is encoded by the encoder.

The downmix processor 440 processes audio data when the number of channels of the sound source is three or more, and includes a decoder, a pair effect synthesizer, a downmixer, and an encoder.

The decoder divides the received sound source into center, left, and right surround low frequency effect channels, that is, six channels.

Next, the pair effect synthesizing unit gives directionality to each channel except the low frequency effect channel through the pair transfer effect synthesis based on the head transfer function, and separates the signal into 10 signal components. It is divided into left and right signal components.

At this time, the head transfer function corresponding to the virtual sound source position condition set in the condition setting unit 460 is used, and in general, the center 0 degrees, the left / right signal ± 30 degrees, the surround left / right signal ± 120 degrees head transfer function is used.

The downmixing unit combines the left and right signal components of each of the 10 signal components and the low frequency effect channel and adds the left and right signal components to an appropriate size to make two channels. The two channel audio data is then encoded by the encoder.

Audio data down to two channels through the audio data processing unit is output as it is divided into left and right channels as the output means is a headphone, and if the output means is a speaker, the signal canceling effect removal process by the crosstalk processing unit 450 Go through

,

입력 좌우 신호를

,

이라고 할 때, 상기 변수와 머리전달함수(HRTF,식에서는 H)와의 관계는 다음과 같다.As shown in Fig. 3, the left and right sound signals finally heard in the human ear are

,

Input left and right signals

,

The relationship between the variable and the head transfer function (HRTF, H in the equation) is as follows.

Y = HX, i.e.

In this case, crosstalk cancellation means that the signal heard in the user's ear through the headphone and the sound signal heard in the user's ear through the speaker are almost equal. In addition, since the sound output through the headphone is directly transmitted to the user's ear, the sound signal transmitted through the headphone output sound and the user's ear through the headphone is about the same.

In other words, crosstalk cancellation is arranged to make the headphone output sound signal equal to the sound signal transmitted to the user's ear through the speaker.

,

) 로 두면,Therefore, the sound signal output from the headphones is B (

,

),

The input signal X is obtained such that the sound signal Y transmitted to the user's ear becomes B. In other words

일 때,

가 되므로 사용자 귀에 헤드폰 출력 사운드 신호와 같은 신호가 전달되어, 신호 상쇄효과가 제거된다.Signal input to the speaker

when,

This results in a signal such as a headphone output sound signal being delivered to the user's ear, eliminating signal cancellation.

를 컨벌루션(CONVOLUTION)하여 크로스토크를 제거한다.In summary, when the user's sound output means is a headphone, two-channel audio data processed by the audio data processor is divided into left and right channels and output as it is. When the sound output means of the user is a speaker, the two-channel audio data processed by the audio data processor is divided into left and right channels and then applied to each channel signal.

CONVOLUTION to eliminate crosstalk.

The head transfer function used at this time is also determined according to the speaker position set in the condition setting unit 460, and a head transfer function corresponding to a 30 degree angle position from the user is often used.

Hereinafter, an audio providing method of an audio providing system according to an exemplary embodiment of the present invention will be described in detail with reference to FIG. 4.

When the user designates a specific virtual sound source position condition, the condition setting unit 460 sets a user specified virtual sound source position condition (S10). Also, the user specifies whether the audio data output means is a speaker or a headphone, and this condition is also set in the condition setting unit 460 (S20).

When the set type of output means is a headphone, the sound source received from the audio source provider is 2-channel downmixed to the audio data processor (S70) and then delivered to the user's ear through the headphone (S80).

On the other hand, if the type of output means is a speaker, the position of the speaker from the user is additionally set in the condition setting unit 460 (S30). At this time, the virtual sound source position condition, the speaker position condition, etc. can be changed by the user in real time setting.

Similarly to the case where the next output means is a headphone, the sound source received from the audio source providing unit is two-channel downmixed by the audio data processing unit (S40), but has an additional crosstalk removing process (S50).

Finally, the sound signal that has undergone the crosstalk removal process is finally output through the speaker (S60).

At this time, as shown in Fig. 5, the two-channel downmix of the audio data through the audio data processing unit is irrelevant to the type of output means set.

That is, in step S100, when a sound signal is input from the audio data providing unit, the number of channels of the input audio signal is checked. In this case, when the number of channels of the input audio signal is 3 or more, it is decoded into multi-channels (S110). In this figure, a case where the 5.1-channel input audio signal is decoded into 6 channels will be described as an example.

That is, if the input audio signal is decoded into the center, left and right, surround left and low frequency effect channels (S110), the directionality is given to the five channels except for the non-directional low frequency effect channel using the head transfer function and 10 signal components are obtained. Separated by. The head transfer function used in this case is the head transfer function corresponding to the virtual sound source position condition specified by the user. In general, the head transfer function is generally 0 degrees, ± 30 degrees for left and right signals, and ± 120 for surround left and right signals. The head transfer function corresponding to the figure is used (S111 ~ S117, S120 ~ S125, S130 ~ S135, S140 ~ S145, S150 ~ S155, S160 ~ S165).

The low frequency effect channel is simply divided into left and right signal components without consideration of directionality (S160 to `165).

Next, each of the 10 signal components and the left and right signal components of the low frequency effect channel are added to the left and right signal components with appropriate sizes to make two channels (S200 and S210). The two-channel audio data is encoded (S220, S230).

Meanwhile, when the number of channels of the input audio signal is checked at step S100 and the number of channels is 2 or less (S300), the following description will be made based on the case where the input audio signal is 2 channels.

As shown in Fig. 6, that is, the input audio signal is decoded into two left and right channels (S310), and the two channel audio data is expanded to four channel audio data. That is, the decoded left and right two channels are bypassed and the appropriate gain value is multiplied by the left and right channel signals, and the multiplied values are added to generate the surround left and right signals (S311 to S313, S320 to S321, S330 to S335, and S340 to S345).

Next, the directionality based on the head transfer function is given to each of the four channel audio data, and the four channel audio data are separated into eight signal components (S315 to S319, S323 to S327, S337 to S339, and S347 to S349). ). In this case, the head transfer function corresponds to a virtual sound source position condition specified by the user. Generally, the head transfer function corresponds to ± 60 degrees for left / right signals and ± 120 degrees for surround left / right signals. do.

In operation S350 and S351, each of the eight signal components is added to the left and right signal components, and is made of two channels and encoded (S353 to S357).

As shown in FIG. 7, the two channel audio data generated and encoded through the above process is decoded into two left and right channels (S400 to S420).

At this time, when the sound output means of the user is a headphone (S430), the left and right two-channel audio data is output as it is through the output unit 470 (S440 ~ S443). If the user's sound output means is a speaker (S430), the left and right two-channel audio data undergoes a crosstalk removal process (S450 to S467).

That is, crosstalk is eliminated by convolving the inverse function of the head transfer function to each channel signal. The head transfer function used at this time is determined by the speaker position set by the user, and the head transfer function corresponding to the 30 degree position from the user is often used.

The sound signal, which has undergone the crosstalk removal process, is transmitted to the user's speaker through the output unit 470 (S469).

As described above is only an embodiment of the present invention, the scope of the present invention is not limited thereto, and various modifications may be made to the matters obvious to those skilled in the art.

As described above, according to the present invention, since a three-dimensional effect is provided to the input audio signal using a predetermined transfer function based on a virtual sound source position condition specified by the user, the user can enjoy a sound having a three-dimensional effect in a desired direction.

Also, depending on the type of output means, it is possible to determine whether to execute the crosstalk removal function.If you do not need the crosstalk removal function such as when listening to music with headphones, you can prevent the unnecessary functions from being executed. The torque cancel function can be executed.

In addition, when the output means is a speaker, the crosstalk removal function is executed based on the pair of speaker mutual positions set and used by the user, so that crosstalk removal can be made more effectively.

Claims (22)

An audio data providing system connected to a user terminal through a network and providing audio data to the user terminal, A condition setting unit for setting audio data providing conditions including a user specified virtual sound source position condition, an output means type, and a speaker position; When the input audio signal is 2 channels or less, the number of channels is increased by 4 or more, and the input audio signal is obtained by giving a stereoscopic feeling in the direction desired by the user by using a head transfer function predetermined based on the virtual sound source position condition set in the condition setting unit. Outputs audio data, and when the input audio signal is three or more channels, the audio obtained by giving a three-dimensional effect in a desired direction to the input audio signal using a predetermined head transfer function based on the virtual sound source position condition set in the condition setting unit An audio data processor for outputting data; And If the type of output means set in the condition setting unit is a speaker, convolution of the inverse function of the head transfer function to each channel signal of the audio data output from the audio data processing unit based on the speaker position set in the condition setting unit. Thereby, including a crosstalk processing unit for removing the offset effect due to mutual interference, The audio data processor may include: a channel checking unit checking a channel number of an input audio signal; An upmix processing unit for increasing the number of channels to 4 or more to give a stereoscopic effect to the input audio signal and outputting the stereo channel after the number of channels identified by the channel checking unit is 2 or less; And a downmix processing unit for adding a stereoscopic effect to the input audio signal and outputting the stereo channel when the number of channels identified by the channel checking unit is 3 or more. The downmix processor may include: a decoder configured to decode an input audio signal; A pair effect synthesizing unit which provides a three-dimensional effect to the decoded audio signal by using the predetermined transfer function; A downmixing unit for downscaling the audio signal to which two-dimensional effects are given; And an encoder for encoding the audio signal down to two channels, The upmix processor may include a decoder configured to decode an input audio signal; A channel expansion unit for extending the number of channels of the decoded audio signal; A pair effect synthesizing unit for providing a three-dimensional effect to each of the channels of the audio signal using a head transfer function; A downmixing unit for downscaling the audio signal to which two-dimensional effects are given; And an encoder for encoding the audio signal down to two channels. delete delete delete delete delete delete delete In the audio data providing method of the system that is connected to the user terminal through a network to provide audio data to the user terminal, a) setting audio data providing conditions including a user specified virtual sound source position condition, an output means type and a speaker position; b) If the input audio signal is less than 2 channels, the number of channels is enlarged to 4 or more, and according to the set audio data provision conditions, the stereo audio in the direction desired by the user is given by using the head transfer function to the input audio signal, or the input audio signal is 3 Obtaining audio data by providing a three-dimensional effect in a direction desired by a user using a head transfer function to an input audio signal according to the set audio data providing condition when the channel is larger than the channel; And c) If the output means type set in step a) is a speaker, converse the inverse function of the head transfer function to each channel signal of the audio data to eliminate the offset effect due to mutual interference, wherein the speaker set in step a) A crosstalk removing step of canceling the offset effect of the mutual interference based on the position; B) checking the number of channels of the input audio signal; b-2) when the number of channels is less than or equal to 2, extending the number of audio signal channels by 4 or more to give each channel a three-dimensional effect in a desired direction by using a head transfer function, and then outputting down to two channels; And b-3) when the number of channels is 3 or more, giving each channel a three-dimensional effect in a desired direction by using a head transfer function, and then outputting down to two channels, Step b-2) may include b-2-1) decoding an audio signal when the number of audio signal channels is 2 or less; b-2-2) expanding the channel number of the decoded audio signal; b-2-3) imparting stereoscopic effect to each of the audio signal channels by using a head transfer function; b-2-4) bringing down the stereo signal given audio signal to two channels; And b-2-5) encoding an audio signal down to two channels into a stereo channel, Step b-3) includes b-3-1) decoding an input audio signal when the number of audio channels is 3 or more; b-3-2) imparting stereoscopic effect to the decoded audio signal by using a head transfer function; b-3-3) bringing down the stereoscopically given audio signal to two channels; And b-3-4) encoding the audio signal down to two channels. delete delete delete delete delete delete delete delete delete delete delete delete delete

Images