KR102529400B1 - Apparatus and method for providing the audio metadata, apparatus and method for providing the audio data, apparatus and method for playing the audio data - Google Patents

Abstract

an audio metadata generating unit generating audio metadata including channel information of audio raw data; and an audio metadata transmitter configured to transmit the generated audio metadata to an audio data reproducing apparatus.

Description

Apparatus and method for providing audio metadata, apparatus and method for providing audio data, apparatus and method for reproducing audio data

The present invention relates to a method for generating multi-channel audio data and providing the same to an audio data reproducing apparatus, and more particularly, to expressing multi-channel audio data using metadata including channel information of an audio signal. It's about how.

In order to implement multi-channel audio data, information on how many signals are made of audio content and which channels are to be arranged at which positions in space is required. Existing 5.1-channel audio data has a basic condition that a total of six signals are produced and reproduced at positions 0, +30, +110, +250, +330, and null degrees.

With the development of UHDTV technology and the use of more speakers than the 5.1 channels provided by HDTV, research on audio reproduction methods with higher realism is receiving great attention. In addition, there are actually various reproduction methods of 5.1 channels or more.

Regarding the conventional multi-channel audio data representation method, Korean Patent Registration No. 10-0522593 discloses that when a plurality of outputs are added in a multi-channel audio system, the added output is generated to harmonize with the existing multi-channel audio. Therefore, we propose a multi-channel audio implementation method that implements natural stereophonic sound.

Specifically, the method for realizing multi-channel audio includes receiving and decoding an encoded audio stream, and generating multi-channel stereophonic sound using the decoded audio stream. Next, the multi-channel audio implementation method includes generating a television left speaker output and a television right speaker output by using a left stereo channel signal, a right stereo channel signal, and a center channel signal in the generated multi-channel sound output. Provides multi-channel audio with added channels.

However, the conventional multi-channel audio implementation method only describes a configuration for converting 5.1 channels to 7.1 channels, and has a disadvantage in that it is difficult to configure audio channel systems in various ways.

In addition, currently, various methods such as Auro 3D and T.Holman 12.2 (10.2) channels express audio channels in their own way by arranging 6 or more speakers in a location other than a horizontal plane, but a separate additional system such as an A/V receiver is required. It has the downside of being absolutely necessary.

Therefore, a technique capable of expressing audio data having multi-channels in various ways is required.

The present invention provides an audio metadata providing apparatus capable of generating audio metadata including channel information of audio raw data and transmitting the generated audio metadata to an audio reproducing apparatus.

The present invention provides an audio data providing device capable of outputting multi-channel audio in various ways by combining audio raw data and audio metadata including channel information of the audio raw data and transmitting the combined audio metadata to an audio reproducing apparatus.

The present invention provides an audio data reproducing apparatus capable of reproducing raw audio data according to audio metadata by demultiplexing or decoding audio data received from an audio data providing apparatus.

The present invention provides an audio data reproducing apparatus capable of channel compatibility through a method of combining audio channel signals included in audio raw data when a channel included in reproduction environment information is a lower channel than a channel included in audio raw data do.

An apparatus for providing audio metadata according to an embodiment of the present invention includes an audio metadata generating unit generating audio metadata including channel information of audio raw data; and an audio metadata transmission unit that transmits the generated audio metadata to an audio data reproducing apparatus.

An audio data providing apparatus according to an embodiment of the present invention includes an audio metadata generating unit generating audio metadata including channel information of audio raw data; an audio data combining unit combining the raw audio data and the generated audio metadata into one audio data; and an audio data transmitter for transmitting the combined audio data to an audio data reproducing apparatus.

An audio data reproducing apparatus according to an embodiment of the present invention includes an audio data receiving unit receiving audio data from an audio data providing apparatus; an audio data analyzer configured to analyze raw audio data and audio metadata from the received audio data; and an audio data reproducing unit reproducing raw audio data based on the analyzed audio metadata and preset reproduction environment information.

According to one embodiment of the present invention, audio metadata including channel information of raw audio data may be generated and transmitted to an audio reproducing apparatus.

According to an embodiment of the present invention, multi-channel audio can be output in various ways by combining audio raw data and audio metadata including channel information of the audio raw data and transmitting the combined audio metadata to an audio reproducing apparatus.

According to an embodiment of the present invention, audio raw data may be reproduced according to audio metadata by demultiplexing or decoding audio data received from an audio data providing apparatus.

According to one embodiment of the present invention, when a channel included in the reproduction environment information is a lower channel than a channel included in the audio raw data, channel compatibility is possible through a method of combining audio channel signals included in the audio raw data do.

1 is a diagram for explaining the configuration of audio data according to an embodiment of the present invention.
2 is a diagram for explaining an operation in which an audio metadata providing device transmits audio metadata to an audio data reproducing device according to an embodiment of the present invention.
3 is a diagram for explaining an operation in which an audio data providing apparatus transmits audio data to an audio data reproducing apparatus according to an embodiment of the present invention.
4 is a diagram showing the configuration of audio metadata according to an embodiment of the present invention.
5 is a diagram for explaining an embodiment of audio metadata configuration according to an embodiment of the present invention.
6 is a diagram for explaining a standard of audio channel location information according to an embodiment of the present invention.
7 is a diagram in which speakers are arranged in a space based on audio channel location information according to an embodiment of the present invention.
8 is a diagram for explaining another embodiment of the audio metadata configuration of the present invention.
9 is a flowchart illustrating an operation of providing audio metadata by an apparatus for providing audio metadata according to an embodiment of the present invention.
10 is a flowchart illustrating an operation of providing audio data by an audio data providing apparatus according to an embodiment of the present invention.
11 is a flowchart illustrating an operation of reproducing audio data by an audio data reproducing apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The audio metadata providing method according to an embodiment of the present invention may be performed by an audio metadata providing device, and the audio data providing method according to an embodiment of the present invention may be performed by an audio data providing device. Also, the method for reproducing audio data according to an embodiment of the present invention may be performed by an apparatus for reproducing audio data. Like reference numerals in each figure indicate like elements.

1 is a diagram for explaining the configuration of audio data according to an embodiment of the present invention.

Referring to FIG. 1 , audio data 110 may include audio raw data 120 and audio metadata 130 . Audio raw data means an audio signal, and may include audio signals as many as the number of channels. The audio raw data 120 may sequentially store produced audio signals. For example, in the case of 5.1-channel audio data 110, the audio raw data 120 includes L (Left), R (Right), C (Center), LFE (Low Frequency Effect), LS (Left Side), Each signal 140 associated with a right side (RS) channel may be included.

The audio metadata 130 means expression information related to an audio signal, and may include channel information of the audio raw data 120 . A detailed description of this will be described later with reference to FIG. 4 .

2 is a diagram for explaining an operation in which an audio metadata providing device transmits audio metadata to an audio data reproducing device according to an embodiment of the present invention.

Referring to FIG. 2 , an audio metadata providing device 210 may include an audio metadata generator 220 and an audio metadata transmitter 230 .

The audio metadata generating unit 220 may generate audio metadata including channel information of audio raw data. The generated audio metadata may be managed separately from raw audio data and may be coded by the audio metadata providing device 210 .

The audio metadata transmission unit 230 may transmit audio metadata to the audio data reproduction device 250 . According to another embodiment, the audio metadata transmitter 230 may multiplex the audio raw data and audio metadata received from the audio raw data providing device 240 and transmit the multiplexed audio data to the audio data reproducing device 250 . Here, multiplexing means a method of combining several signals and processing them into a single signal.

The audio raw data providing device 240 may transmit audio raw data in which audio signals are aggregated for each channel to the audio data reproducing device 250 . In some cases, the audio metadata providing device 210 may create audio metadata based on audio raw data in conjunction with the audio raw data providing device 240 .

The audio data reproducing apparatus 250 may reproduce the audio raw data by analyzing the received audio raw data and audio metadata. In this case, the audio data reproducing apparatus 250 may reproduce the raw audio data according to the channel information based on the audio environment setting information and the audio metadata stored in the audio data reproducing apparatus 250 .

3 is a diagram for explaining an operation in which an audio data providing apparatus transmits audio data to an audio data reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 3 , the audio data providing device 310 may include an audio metadata generator 320, an audio data combiner 330, and an audio data transmitter 340.

The audio metadata generating unit 320 may generate audio metadata including channel information of audio raw data. That is, the audio metadata generation unit 320 may analyze raw audio data and represent channel information of an audio signal as audio metadata.

The audio data combiner 330 may combine audio raw data and audio metadata into one audio data. At this time, the audio data combiner 330 may code only audio raw data or audio metadata or code both audio raw data and audio metadata.

The audio data transmission unit 340 may transmit the combined audio data to the audio data reproduction device 350 . In other words, the audio data transmission unit 340 may multiplex the coded audio data and transmit the coded audio data to the audio data reproducing apparatus 350 .

The audio data reproducing apparatus 350 may include an audio data receiving unit 360 , an audio data analyzing unit 370 , and an audio data reproducing unit 380 .

The audio data receiving unit 360 may receive audio data from the audio data providing device 310 .

The audio data analyzer 370 may analyze raw audio data and audio metadata in the received audio data. The audio data analyzer 370 may demultiplex or decode audio data. The audio data analyzer 370 may separate audio raw data and audio metadata from audio data through demultiplexing and decoding processes, and may extract an audio signal for each channel from the audio raw data.

According to another embodiment of the present invention, the audio data analyzer 370 separates the received audio data into audio raw data and audio metadata, and the user uses the separated audio raw data and audio metadata to obtain an additional audio signal. can be edited or reconstructed.

The audio data reproducing unit 380 may reproduce raw audio data based on the analyzed audio metadata and preset reproduction environment information. Here, the preset playback environment refers to audio environment setting information of an audio playback device, such as a home TV system or a speaker arrangement situation such as an A/V receiver.

When the channel included in the playback environment information is lower than the channel included in the audio raw data, the audio data reproducing unit 380 outputs the audio included in the audio raw data according to the backward compatibility information of the audio channel included in the audio metadata. Signals of channels can be combined and changed to sub-channels.

For example, if the number of channels included in the raw audio data is 6 and the number of set channels included in the reproduction environment information of the audio reproducing apparatus is 2, the audio data reproducing unit 380 converts the received audio data into 2 channels. Downmixing to lower channels is possible. Here, downmixing means converting audio data obtained by combining several channels into audio data having a lower number of channels. Through this, the audio data reproducing apparatus 350 may reproduce audio data through channel compatibility even when receiving audio data having more channels than the channels included in the audio environment setting information.

When the audio data reproducing apparatus 350 receives audio data having a channel lower than the channel included in the audio environment setting information, it does not output a channel that does not include an audio signal, so that the audio data is output regardless of the number of channels. can be played

4 is a diagram showing the configuration of audio metadata according to an embodiment of the present invention.

Referring to FIG. 4 , audio metadata 410 includes audio channel number information 420, audio channel name information 430, audio channel location information 440, or audio channel backward compatibility information 450. At least one of them may be included.

The information on the number of audio channels 420 indicates the number of audio signals included in the audio raw data. For example, if a 5.1-channel signal is included in the audio raw data, the audio channel number information 420 has a value of 6. Also, if a 7.1-channel signal is included in the audio raw data, the audio channel number information 420 has a value of 8. In some cases, the information on the number of audio channels 420 may indicate the number of objects of an audio signal stored on an object basis.

The audio channel name information 430 represents the channel name of each audio signal included in the audio raw data. For example, in the case of a 5.1-channel signal, the audio channel name information 430 may include (L, R, C, LFE, LS, RS) information. This means that the audio signals stored in the audio raw data correspond to the names of L, R, C, LFE, LS, and RS in sequence. Here, the definition of the name of the audio channel and the order of the audio signal is not compulsory and can be arbitrarily set by the manufacturer. However, for compatibility with the existing system, it is desirable to unify the 5.1-channel and 2.0-channel signals in the form of (L, R, C, LFE, LS, RS) and (L, R), respectively.

The location information 440 of the audio channel indicates where the audio signal channel should be located spatially. The location information 440 of the audio channel may include horizontal azimuth information and vertical azimuth information.

As shown in FIG. 6, when the user 610 looks at the front, the horizontal azimuth is set at 0 degrees 620 as the reference point based on the horizontal surface on the ground, and the clockwise direction is the (+) direction. can be set Vertical azimuth angle As shown in FIG. 6, when the user 630 looks at the front, the front is set as the reference point 0 degree 640 based on the plane perpendicular to the ground, and the upward direction is the (+) direction. can be set

For example, in the case of a 2.0 channel, if the name information 430 of the audio channel is expressed as (L, R) and the location information 440 of the audio channel is expressed as [(330, 0), (30, 0)] . If it does not matter where it is placed, it can be displayed using a null symbol, and it can be expressed as (null, null) in an azimuth.

The audio channel backwards compatibility information 450 represents information on a method for combining audio channel signals included in audio raw data and changing them to lower channels. For example, the backward compatibility information 450 may indicate how audio raw data having a 7.1 channel is compatible with a 5.1 channel or a 2.0 channel. Audio data having a number of channels greater than 5.1 may include information on a change method to 5.1 or 2.0, and 5.1-channel audio data only needs to include information about a compatible method to 2.0.

5 is a diagram for explaining an embodiment of audio metadata configuration according to an embodiment of the present invention.

According to FIG. 5, audio metadata 510 includes audio channel number information 520, audio channel name information 530, audio channel location information 540, and backward compatibility information 550 of audio channels. are doing

The value of the information on the number of audio channels 520 is 6, indicating that the number of channels of the audio raw data is 6 and the LFE channel is 1, which is 5.1 channels. In addition, according to the name information 530 of the audio channel, it can be seen that each channel name of the audio signals included in the audio raw data is L, R, C, LFE, LS, and RS in sequence.

Looking at the audio channel location information 540, the L channel becomes (330, 0) and is placed at 330 degrees horizontally and 0 degrees vertically, and the R channel becomes (30, 0) and is placed at 30 degrees horizontally and 0 degrees vertically. . The C channel is positioned at (0, 0) at 0 degrees horizontally and 0 degrees vertically, and the LS channel is positioned at (250, 0) at 250 degrees horizontally and 0 degrees vertically. In addition, the RS channel becomes (110, 0) and is arranged at 110 degrees horizontally and 0 degrees vertically, and the LFE channel becomes (null, null), indicating that it does not matter where it is arranged. Audio channel location information of 5.1-channel and 2.0-channel signals are [(330, 0), (30, 0), (0, 0), (null, null), (250, 0) respectively for compatibility with the existing system. , (110, 0)], [(330, 0), (30, 0)].

An implementation of this in real space is shown in FIG. 7 . FIG. 7 is a diagram in which speakers are arranged based on the audio channel location information 540 in FIG. 5 .

Referring to FIG. 7 , with the user 710 as the center, a speaker is positioned at a position where each channel is to be placed. This represents multi-channel audio data based on information included in audio metadata, and it is possible to implement multi-channel audio data having more channels than 5.1 channels in various ways using this.

In FIG. 5 , looking at backward compatibility information 550 of an audio channel, it is possible to know a constant 560 and a calculation formula 570 for downmixing from 5.1 channels to 2.0 channels. The constant (a, k) 560 is an arbitrary constant created by the audio data producer, and the calculation formula 570 can be stored according to the lower channel order. In the calculation formula 570, D(1) means the first data a of (D), and D(2) means the second data k of (D). N(1) means an L channel signal as the first data of (N), and N(2) means an R channel signal as the second data of (N). Subsequently, N(3) represents the C channel signal, N(4) represents the LFE channel signal, N(5) represents the LS channel signal, and N(6) represents the RS channel signal. That is, the backward compatibility information 550 of the audio channel may store downmixing information as shown in Table 1 below. The above embodiment is only an example of downmixing from 5.1 channels to 2.0 channels, and various modifications and arbitrary settings are possible.

channel layout channel matrixing
(a and k are arbitrary constants) 5.1 -> 2.0

8 is a diagram for explaining another embodiment of the audio metadata configuration of the present invention.

According to FIG. 8, audio metadata 810 includes audio channel number information 820, audio channel name information 830, audio channel location information 840, and audio channel backward compatibility information 850. It contains compatibility information for three channels.

The value of the information on the number of audio channels 820 is 12, indicating that the number of channels of the audio raw data is 12 and the number of LFE channels is 2, which is 10.2 channels. In addition, according to the audio channel name information 830, each channel name of the audio signals included in the audio raw data is sequentially L, R, C, LH, RH, LS, RS, LB, RB, TC, LFE1, LFE2 It can be seen that

Looking at the audio channel location information 840, the L channel is (330, 0) and is placed at 330 degrees horizontally and 0 degrees vertically, and the R channel is (30, 0) at 30 degrees horizontally and 0 degrees vertically, The C channel is (0, 0), which is placed at 0 degrees horizontally and 0 degrees vertically. Subsequently, the LH channel becomes (330, 30) and is placed at 330 degrees horizontally and 30 degrees vertically, the RH channel is placed at (30, 30) at 30 degrees horizontally and 30 degrees vertically, and the LS channel is (270, 0), which is placed at 270 degrees horizontally and 0 degrees vertically. In addition, the RS channel becomes (90, 0) and is placed at 90 degrees horizontally and 0 degrees vertically, the LB channel is placed at (210, 0) horizontally 210 degrees and 0 degrees vertically, and the RB channel is placed at (150, 0) ) at 150 degrees horizontally and 0 degrees vertically. The TC channel is positioned at (0, 90) at 0 degrees horizontally and 90 degrees vertically. In this way, the audio metadata 810 can include audio channel information in a layer higher than the horizontal plane as well as the horizontal plane, so that the audio producer can construct a stereoscopic sound system.

Looking at the backward compatibility information 850 of the audio channel, constants 860 and formulas 870 for downmixing from 10.2 channels to 7.1 channels, 5.1 channels, and 2.0 channels can be found. The constants 860 (a1, c1), (a2, c2), (a3, c3) are arbitrary constants created by the audio data producer, and the downmixing formula information 870 may be stored according to the order of sub-channels. Each downmixing formula information may include signal combination and conversion information for changing from 10.2 channels to each channel. As a result, even when the audio reproducing apparatus receives 10.2-channel audio data, the audio data can be reproduced compatible with the channel configuration of the audio reproducing apparatus. The order in which downmixing formula information is stored can be arbitrarily set by a producer, and is not limited to the above embodiment.

9 is a flowchart illustrating an operation of providing audio metadata by an apparatus for providing audio metadata according to an embodiment of the present invention.

In operation 910, the audio metadata providing apparatus may generate audio metadata including information related to audio raw data. Audio metadata generated by the audio metadata providing device may be managed separately from audio raw data and may be coded by the audio metadata providing device.

In step 920, the audio metadata providing device may transmit the generated audio metadata to the audio data reproducing device. According to another embodiment, the audio metadata providing device may multiplex audio raw data and audio metadata received from the audio raw data providing device and transmit the multiplexed audio data to the audio data reproducing device.

10 is a flowchart illustrating an operation of providing audio data by an audio data providing apparatus according to an embodiment of the present invention.

In operation 1010, the audio data providing apparatus may generate audio metadata including information related to audio raw data. That is, the apparatus for providing audio data may represent information related to an audio signal as audio metadata based on raw audio data.

In step 1020, the audio data providing device may combine the audio raw data and the generated audio metadata into one audio data. In this case, the audio data providing apparatus may separately code the audio raw data and the audio metadata or code the audio raw data and the audio metadata together.

In step 1030, the audio data providing device may transmit the combined audio data to the audio data reproducing device. The audio data providing device may multiplex the coded audio data and transmit the coded audio data to the audio data reproducing device.

11 is a flowchart illustrating an operation of reproducing audio data by an audio data reproducing apparatus according to an embodiment of the present invention.

In step 1110, the audio data reproduction device may receive audio data from the audio data providing device.

In step 1120, the audio data reproducing apparatus may analyze audio raw data and audio metadata from the received audio data. An audio data reproducing apparatus may demultiplex or decode audio data. The audio reproducing apparatus may separate audio raw data and audio metadata from audio data through demultiplexing and decoding processes, and may extract an audio signal for each channel from the audio raw data.

According to another embodiment of the present invention, the audio data reproducing apparatus separates the received audio data into audio raw data and audio metadata, and the user uses the separated audio raw data and audio metadata to edit or edit an additional audio signal. can reconstruct

In step 1130, the audio data reproducing apparatus, when the channel included in the reproduction environment information is lower than the channel included in the audio raw data, converts the audio data to the audio raw data according to the backward compatible information of the audio channel included in the audio metadata. Signals of included audio channels can be combined and changed to sub-channels.

In step 1140, the audio data reproducing apparatus, when the channel included in the reproduction environment information is the same as or higher than the channel included in the raw audio data, based on the analyzed audio metadata and the preset reproduction environment information, Audio raw data can be reproduced. Also, the audio data reproducing apparatus may reproduce audio raw data changed to a lower channel in step 1130.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. A computer readable medium is any data storage device capable of storing data that can be later read by a computer system. Examples of computer readable media include read only memory, random access memory, CD-ROM, DVD, magnetic tape, optical data storage devices, and carrier waves. Computer readable media can also be distributed across network-connected computer systems, such that computer readable code is stored and executed in a distributed manner. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software.

As described above, although the present invention has been described by the limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions. this is possible

Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, but should be defined by not only the claims to be described later, but also those equivalent to these claims.

110: audio data
120: audio raw data
130: audio metadata
140: audio signal

Claims (10)

In the audio data providing method of the audio data providing device,
generating information related to a channel of audio data;
providing the audio data and information related to the channel to an audio data reproducing apparatus;
including,
The audio data is
wherein the audio data is reproduced based on information related to the channel and a reproduction environment when the audio data is reproduced in the audio data reproducing apparatus. According to claim 1,
Information related to the channel,
A method of providing audio data including name information of the channel, location information of the channel, and backward compatibility information of the channel. In the audio data reproduction method of the audio data reproduction device,
receiving audio data and information related to a channel of the audio data from an audio data providing device;
Reproducing the audio data using information related to a channel of the audio data
including,
The audio data is
wherein the audio data reproducing apparatus reproduces the audio data based on information related to the channel and a reproduction environment when the audio data is reproduced. According to claim 3,
Information related to the channel,
A method of reproducing audio data including name information of the channel, location information of the channel, and backward compatibility information of the channel. According to claim 3,
The information related to the channel further includes information on the number of audio channels,
The information on the number of channels,
How to reproduce audio data, which is the number of channels associated with audio data. According to claim 3,
Location information of the channel,
The method of reproducing audio data, which is information indicating where the channel should be located spatially. According to claim 3,
Location information of the channel,
A method of reproducing audio data including horizontal azimuth information and vertical azimuth information. According to claim 3,
Name information of the channel,
A method of reproducing audio data, which is a name of each of channels for audio data. According to claim 4,
The backward compatibility information of the channel,
A method of reproducing audio data that is information on a method for changing the audio data to a lower channel using a signal of a channel for the audio data. delete

Images