KR20220030983A - Apparatus and method for providing the audio metadata, apparatus and method for providing the audio data, apparatus and method for playing the audio data - Google Patents

Abstract

Disclosed is an apparatus for providing audio metadata. The apparatus comprises: an audio metadata generation unit generating audio metadata in which channel information of audio raw data is included; and an audio metadata transmission unit transmitting the generated audio metadata to an apparatus for playing audio data.

Description

Apparatus and method for providing audio metadata, apparatus and method for providing audio data, apparatus and method for playing audio data

The present invention relates to a method of generating multi-channel audio data and providing it to an audio data reproducing apparatus, and more particularly, to a method of expressing multi-channel audio data using metadata including channel information of an audio signal. it's about how

In order to implement audio data in a multi-channel format, information on how many signals are made of audio content and which channels are to be placed at which positions in space is required. Existing 5.1-channel audio data is based on the basic condition that a total of 6 signals are produced and reproduced at positions 0, +30, +110, +250, +330, and null.

With the development of UHDTV technology and the use of more speakers than 5.1 channels provided by HDTV, research on a more realistic audio reproduction method is receiving great attention. In addition, there are actually various reproduction methods of 5.1 channels or more.

In relation to the conventional method of representing audio data having multi-channels, Korean Patent Registration No. 10-0522593 discloses that when a plurality of outputs are added in a multi-channel audio system, the added outputs are created to harmonize with the existing multi-channel audio. Therefore, we propose a multi-channel audio implementation method that realizes natural stereophonic sound.

Specifically, the method for implementing multi-channel audio includes receiving and decoding an encoded audio stream, and generating multi-channel stereophonic sound using the decoded audio stream. Next, the multi-channel audio implementation method includes generating a television left speaker output and a television right speaker output by using a left stereo channel signal, a right stereo channel signal, and a center channel signal in the generated multi-channel sound output, Provides multi-channel audio with added channels.

However, the conventional method for implementing multi-channel audio only describes a configuration for converting 5.1 channels to 7.1 channels, and there is a disadvantage in that it is difficult to configure an audio channel system using various methods.

In addition, various methods such as Auro 3D and T.Holman 12.2 (10.2) channels currently express audio channels in their own way by arranging 6 or more speakers in locations other than the horizontal plane, but additional systems such as A/V receivers are not available. The downside is that it is absolutely necessary.

Accordingly, there is a need for a technology capable of expressing multi-channel audio data in various ways.

The present invention provides an audio metadata providing apparatus capable of generating audio metadata including channel information of raw audio data and transmitting the audio metadata to an audio reproducing apparatus.

The present invention provides an audio data providing apparatus capable of outputting multi-channel audio in various ways by combining raw audio data and audio metadata including channel information of the audio raw data and transmitting it to an audio reproducing apparatus.

The present invention provides an audio data reproducing apparatus capable of reproducing audio raw data according to audio metadata by demultiplexing or decoding audio data received from an audio data providing apparatus.

The present invention provides an audio data reproducing apparatus capable of channel compatibility through a method of combining signals of audio channels included in raw audio data when a channel included in playback environment information is a lower channel than a channel included in raw audio data do.

According to an embodiment of the present invention, an apparatus for providing audio metadata includes an audio metadata generator for generating audio metadata including channel information of raw audio data; and an audio metadata transmitter for transmitting the generated audio metadata to the audio data reproducing apparatus.

According to an embodiment of the present invention, an apparatus for providing audio data includes an audio metadata generator for generating audio metadata including channel information of raw audio data; an audio data combining unit combining the audio raw data and the generated audio metadata into one audio data; and an audio data transmitter for transmitting the combined audio data to the audio data reproducing apparatus.

An audio data reproducing apparatus according to an embodiment of the present invention includes: an audio data receiving unit for receiving audio data from an audio data providing apparatus; an audio data analyzer for analyzing raw audio data and audio metadata from the received audio data; and an audio data reproducing unit that reproduces raw audio data based on the analyzed audio metadata and preset reproduction environment information.

According to an embodiment of the present invention, audio metadata including channel information of raw audio data may be generated and transmitted to the audio reproducing apparatus.

According to an embodiment of the present invention, multi-channel audio may be output in various ways by combining raw audio data and audio metadata including channel information of raw audio data, and transmitting it to an audio reproducing apparatus.

According to an embodiment of the present invention, audio data received from an audio data providing apparatus may be demultiplexed or decoded to reproduce raw audio data according to audio metadata.

According to an embodiment of the present invention, when a channel included in the reproduction environment information is a lower channel than a channel included in the audio raw data, channel compatibility is possible through a method of combining signals of an audio channel included in the audio raw data Do.

1 is a diagram for explaining the configuration of audio data according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining an operation in which an audio metadata providing apparatus transmits audio metadata to an audio data reproducing apparatus according to an embodiment of the present invention.
FIG. 3 is a diagram for explaining an operation in which an audio data providing apparatus transmits audio data to an audio data reproducing apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a configuration of audio metadata according to an embodiment of the present invention.
FIG. 5 is a diagram for explaining an embodiment of an audio metadata configuration according to an embodiment of the present invention.
6 is a diagram for explaining a standard of audio channel location information according to an embodiment of the present invention.
7 is a diagram in which a speaker is arranged in a space based on audio channel position information according to an embodiment of the present invention.
8 is a diagram for explaining another embodiment of the audio metadata configuration of the present invention.
9 is a flowchart illustrating an operation of providing audio metadata by an apparatus for providing audio metadata according to an embodiment of the present invention.
10 is a flowchart illustrating an operation of providing audio data by an audio data providing apparatus according to an embodiment of the present invention.
11 is a flowchart illustrating an operation in which an audio data reproducing apparatus reproduces audio data according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The method for providing audio metadata according to an embodiment of the present invention may be performed by an apparatus for providing audio metadata, and the method for providing audio data according to an embodiment of the present invention may be performed by an apparatus for providing audio data. Also, the audio data reproducing method according to an embodiment of the present invention may be performed by an audio data reproducing apparatus. Like reference numerals in each figure indicate like elements.

1 is a diagram for explaining the configuration of audio data according to an embodiment of the present invention.

Referring to FIG. 1 , audio data 110 may include raw audio data 120 and audio metadata 130 . Audio raw data means an audio signal, and may include as many audio signals as the number of channels. The audio raw data 120 may sequentially store the produced audio signals. For example, in the case of 5.1-channel audio data 110, the audio raw data 120 includes L (Left), R (Right), C (Center), LFE (Low Frequency Effect), LS (Left Side), Each signal 140 associated with a right side (RS) channel may be included.

The audio metadata 130 refers to expression information related to an audio signal, and may include channel information of the audio raw data 120 . A detailed description thereof will be described later with reference to FIG. 4 .

FIG. 2 is a diagram for explaining an operation in which an audio metadata providing apparatus transmits audio metadata to an audio data reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 2 , the audio metadata providing apparatus 210 may include an audio metadata generating unit 220 and an audio metadata transmitting unit 230 .

The audio metadata generator 220 may generate audio metadata including channel information of raw audio data. The generated audio metadata may be managed separately from the audio raw data, and may be coded by the audio metadata providing apparatus 210 .

The audio metadata transmitter 230 may transmit audio metadata to the audio data reproducing apparatus 250 . According to another embodiment, the audio metadata transmitter 230 may multiplex the raw audio data and the audio metadata received from the audio raw data providing apparatus 240 and transmit the multiplexed to the audio data reproducing apparatus 250 . Here, multiplexing refers to a method of combining several signals and processing them into one signal.

The audio raw data providing apparatus 240 may transmit the audio raw data in which audio signals are aggregated for each channel to the audio data reproducing apparatus 250 . In some cases, the audio metadata providing apparatus 210 may work with the audio raw data providing apparatus 240 to generate audio metadata based on the audio raw data.

The audio data reproducing apparatus 250 may reproduce the audio raw data by analyzing the received audio raw data and audio metadata. In this case, the audio data reproducing apparatus 250 may reproduce the audio raw data according to the channel information based on the audio environment setting information and the audio metadata stored in the audio data reproducing apparatus 250 .

FIG. 3 is a diagram for explaining an operation in which an audio data providing apparatus transmits audio data to an audio data reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 3 , the audio data providing apparatus 310 may include an audio metadata generating unit 320 , an audio data combining unit 330 , and an audio data transmitting unit 340 .

The audio metadata generator 320 may generate audio metadata including channel information of raw audio data. That is, the audio metadata generator 320 may analyze raw audio data and represent channel information of an audio signal as audio metadata.

The audio data combiner 330 may combine raw audio data and audio metadata into one audio data. In this case, the audio data combining unit 330 may code only the audio raw data or audio metadata, or both the audio raw data and the audio metadata.

The audio data transmitter 340 may transmit the combined audio data to the audio data reproducing apparatus 350 . In other words, the audio data transmitter 340 may multiplex the coded audio data and transmit it to the audio data reproducing apparatus 350 .

The audio data reproducing apparatus 350 may include an audio data receiving unit 360 , an audio data analyzing unit 370 , and an audio data reproducing unit 380 .

The audio data receiver 360 may receive audio data from the audio data providing apparatus 310 .

The audio data analyzer 370 may analyze raw audio data and audio metadata from the received audio data. The audio data analyzer 370 may demultiplex or decode audio data. The audio data analyzer 370 may separate raw audio data and audio metadata from audio data through demultiplexing and decoding processing, and may extract an audio signal for each channel from the audio raw data.

According to another embodiment of the present invention, the audio data analyzer 370 separates the received audio data into audio raw data and audio metadata, and the user uses the separated audio raw data and audio metadata to obtain an additional audio signal. can be edited or reconstructed.

The audio data reproduction unit 380 may reproduce the audio raw data based on the analyzed audio metadata and preset reproduction environment information. Here, the preset reproduction environment refers to audio environment setting information of the audio reproduction device, such as a situation in which speakers are arranged in a TV system at home or an A/V receiver.

When the channel included in the reproduction environment information is a lower channel than the channel included in the audio raw data, the audio data reproducing unit 380 is configured to configure the audio included in the raw audio data according to backward compatibility information of the audio channel included in the audio metadata. It can be changed to a sub-channel by combining the signals of the channels.

For example, when the number of channels included in the audio raw data is 6 and the set number of channels included in the reproduction environment information of the audio reproducing apparatus is 2, the audio data reproducing unit 380 converts the received audio data to two channels. You can downmix to a lower channel. Here, downmixing refers to converting audio data in which multiple channels are combined into audio data having a low number of channels. Through this, the audio data reproducing apparatus 350 may reproduce the audio data through channel compatibility even when receiving audio data having more channels than the channels included in the audio environment setting information.

When the audio data reproducing apparatus 350 receives audio data having a lower channel than the channel included in the audio environment setting information, the audio data reproducing apparatus 350 does not output the channel not including the audio signal, so that the audio data is played regardless of the number of channels. can play

4 is a diagram illustrating a configuration of audio metadata according to an embodiment of the present invention.

Referring to FIG. 4 , audio metadata 410 includes information on the number of audio channels 420 , name information of audio channels 430 , location information 440 of audio channels, or backward compatibility information 450 on audio channels. may include at least one of

The audio channel number information 420 indicates the number of audio signals included in the audio raw data. For example, if a 5.1-channel signal is included in the audio raw data, the audio channel number information 420 has a value of 6. Also, if the 7.1-channel signal is included in the audio raw data, the audio channel number information 420 has a value of 8. In some cases, the information on the number of audio channels 420 may indicate the number of objects of the audio signal stored on an object basis.

The audio channel name information 430 indicates the channel name of each audio signal included in the audio raw data. For example, in the case of a 5.1-channel signal, the audio channel name information 430 may include (L, R, C, LFE, LS, RS) information. This means that the audio signals stored in the audio raw data correspond to the names of L, R, C, LFE, LS, and RS in sequence. Here, the definition of the audio channel name and audio signal order is not compulsory, but can be set arbitrarily by the producer. However, for compatibility with the existing system, it is desirable to use the 5.1-channel and 2.0-channel signals unified in the form (L, R, C, LFE, LS, RS) and (L, R), respectively.

The location information 440 of the audio channel indicates where the audio signal channel should be spatially disposed. The location information 440 of the audio channel may include horizontal azimuth information and vertical azimuth information.

As for the horizontal azimuth, when the user 610 looks at the front as in FIG. 6 , the front is the reference point 0 degrees 620 with respect to the plane horizontal to the ground, and the clockwise direction is the (+) direction. can be set. Vertical azimuth When the user 630 looks to the front as in FIG. 6 , the front is the reference point 0 degrees (640) with respect to the plane perpendicular to the ground, and the upward direction is the (+) direction. can be set.

For example, in the case of 2.0 channels, if the audio channel name information 430 is (L, R) and the audio channel location information 440 is expressed as [(330, 0), (30, 0)] , indicates that the L channel is arranged at 330 degrees horizontally and 0 degrees vertically, and the R channel is arranged at 30 degrees horizontally and 0 degrees vertically. In the case where it does not matter where it is placed, it can be displayed using a null symbol, and can be expressed as (null, null) in an azimuth angle.

The backward compatibility information 450 of the audio channel indicates information on a method of changing the audio channel signal included in the audio raw data into a lower channel by combining the audio channel signal. For example, the backward compatibility information 450 may indicate how raw audio data having 7.1 channels can be compatible with 5.1 channels or 2.0 channels. Audio data having a number of channels greater than 5.1 channels may include information on how to change to 5.1 or 2.0 channels, and 5.1-channel audio data only needs to include information on compatibility to 2.0 channels.

5 is a diagram for explaining an embodiment of an audio metadata configuration according to an embodiment of the present invention.

According to FIG. 5 , audio metadata 510 includes information on the number of audio channels 520 , name information 530 on audio channels, location information 540 on audio channels, and backward compatibility information 550 on audio channels. are doing

The value of the audio channel number information 520 is 6, indicating that the audio raw data has 6 channels and 1 LFE channel, which is 5.1 channels. Also, according to the audio channel name information 530 , it can be seen that the channel names of the audio signals included in the audio raw data are L, R, C, LFE, LS, and RS in turn.

Looking at the audio channel location information 540, the L channel becomes (330, 0) and is arranged at 330 degrees horizontally and 0 degrees vertically, and the R channel becomes (30, 0) and is arranged at 30 degrees horizontally and 0 degrees vertically. . The C channel is (0, 0) at 0 degrees horizontally and 0 degrees vertically, and the LS channel is (250, 0) at 250 degrees horizontally and 0 degrees vertically. In addition, the RS channel becomes (110, 0) and is arranged at 110 degrees horizontally and 0 degrees vertically, and the LFE channel becomes (null, null) to indicate that it does not matter where it is arranged. Audio channel position information of 5.1 and 2.0 channel signals is [(330, 0), (30, 0), (0, 0), (null, null), (250, 0), respectively, for compatibility with existing systems. , (110, 0)], [(330, 0), (30, 0)].

The realization of this in real space is shown in FIG. 7 . FIG. 7 is a diagram in which speakers are arranged based on the audio channel location information 540 of FIG. 5 .

Referring to FIG. 7 , a speaker is located at a position where each channel is to be arranged, centering on the user 710 . This represents multi-channel audio data based on information included in audio metadata, and using this, multi-channel audio data having more channels than 5.1 channels can be implemented in various ways.

Referring to the backward compatibility information 550 of the audio channel in FIG. 5 , a constant 560 and a calculation formula 570 for downmixing from a 5.1 channel to a 2.0 channel can be found. A constant (a, k) 560 is an arbitrary constant created by an audio data producer, and the formula 570 may be stored according to a sub-channel order. In Equation 570, D(1) means a, which is the first data of (D), and D(2) means k, which is the second data of (D). N(1) is the first data of (N) and means an L-channel signal, and N(2) is the second data of (N), which means an R-channel signal. Subsequently, N(3) denotes a C channel signal, N(4) denotes an LFE channel signal, N(5) denotes an LS channel signal, and N(6) denotes an RS channel signal. That is, the backward compatibility information 550 of the audio channel may store downmixing information as shown in Table 1 below. The above-described embodiment is merely an example of downmixing from a 5.1 channel to a 2.0 channel, and various modifications and arbitrary settings are possible.

Channel Layout Channel Matrixing
(a and k are arbitrary constants) 5.1 -> 2.0

8 is a diagram for explaining another embodiment of the audio metadata configuration of the present invention.

According to FIG. 8 , audio metadata 810 includes audio channel number information 820 , audio channel name information 830 , audio channel location information 840 , and audio channel backward compatibility information 850 . It contains compatibility information to three channels.

The value of the audio channel number information 820 is 12, indicating that the number of channels of the audio raw data is 12 and there are two LFE channels, 10.2 channels. In addition, according to the audio channel name information 830 , the respective channel names of the audio signals included in the audio raw data are sequentially L, R, C, LH, RH, LS, RS, LB, RB, TC, LFE1, LFE2. it can be seen that

Looking at the audio channel location information 840, the L channel becomes (330, 0) and is arranged at 330 degrees horizontally and 0 degrees vertically, and the R channel is (30, 0) at 30 degrees horizontally and 0 degrees vertically, Channel C is (0, 0) arranged at 0 degrees horizontally and 0 degrees vertically. Subsequently, the LH channel becomes (330, 30) and is arranged at 330 degrees horizontally and 30 degrees vertically, the RH channel is (30, 30) at 30 degrees horizontally and 30 degrees vertically, and the LS channel is (270, 0) at 270 degrees horizontally and 0 degrees vertically. Also, the RS channel becomes (90, 0) and is arranged at 90 degrees horizontally and 0 degrees vertically, the LB channel is (210, 0) at 210 degrees horizontally and 0 degrees vertically, and the RB channel is (150, 0). ) at 150 degrees horizontally and 0 degrees vertically. The TC channel is (0, 90) at 0 degrees horizontally and 90 degrees vertically, and the LFE1 channel and LEF2 channel become (null, null), indicating that it does not matter where they are placed. As described above, since the audio metadata 810 can include audio channel information in a layer higher than the horizontal plane as well as the horizontal plane, the audio producer can configure a three-dimensional sound system.

Looking at the backward compatibility information 850 of the audio channel, a constant 860 and a calculation formula 870 for downmixing from 10.2 channel to 7.1 channel, 5.1 channel, and 2.0 channel can be seen. The constants 860 (a1, c1), (a2, c2) (a3, c3) are arbitrary constants created by an audio data producer, and the downmixing formula information 870 may be stored according to a sub-channel order. Each downmixing formula information may include signal combination and conversion information for changing from 10.2 channel to each channel. Accordingly, even if the audio reproducing apparatus receives the 10.2-channel audio data, the audio data may be reproduced compatible with the channel configuration of the audio reproducing apparatus. The order in which the downmixing formula information is stored may be arbitrarily set by the manufacturer, and is not limited to the above embodiment.

9 is a flowchart illustrating an operation of providing audio metadata by an apparatus for providing audio metadata according to an embodiment of the present invention.

In operation 910, the audio metadata providing apparatus may generate audio metadata including information related to raw audio data. Audio metadata generated by the audio metadata providing apparatus may be managed separately from audio raw data, and may be coded by the audio metadata providing apparatus.

In operation 920, the audio metadata providing apparatus may transmit the generated audio metadata to the audio data reproducing apparatus. According to another embodiment, the audio metadata providing apparatus may multiplex audio raw data and audio metadata received from the audio raw data providing apparatus and transmit the multiplexed audio data to the audio data reproducing apparatus.

10 is a flowchart illustrating an operation of providing audio data by an audio data providing apparatus according to an embodiment of the present invention.

In operation 1010, the audio data providing apparatus may generate audio metadata including information related to raw audio data. That is, the audio data providing apparatus may represent information related to the audio signal as audio metadata based on the audio raw data.

In operation 1020, the audio data providing apparatus may combine the audio raw data and the generated audio metadata into one audio data. In this case, the audio data providing apparatus may separately code the audio raw data and the audio metadata or encode the audio raw data and the audio metadata together.

In step 1030, the audio data providing apparatus may transmit the combined audio data to the audio data reproducing apparatus. The audio data providing apparatus may multiplex the coded audio data and transmit it to the audio data reproducing apparatus.

11 is a flowchart illustrating an operation in which an audio data reproducing apparatus reproduces audio data according to an embodiment of the present invention.

In operation 1110, the audio data reproducing apparatus may receive audio data from the audio data providing apparatus.

In operation 1120, the audio data reproducing apparatus may analyze raw audio data and audio metadata from the received audio data. The audio data reproducing apparatus may demultiplex or decode audio data. The audio reproducing apparatus may separate raw audio data and audio metadata from audio data through demultiplexing and decoding processing, and may extract an audio signal for each channel from the audio raw data.

According to another embodiment of the present invention, the audio data reproducing apparatus separates the received audio data into audio raw data and audio metadata, and the user uses the separated audio raw data and audio metadata to edit or edit an additional audio signal. can be reconstructed

In step 1130, when the channel included in the playback environment information is a lower channel than the channel included in the audio raw data, the audio data reproducing apparatus provides data to the audio raw data according to backward compatibility information of the audio channel included in the audio metadata. It can be changed to a sub-channel by combining the signals of the included audio channels.

In step 1140, when the channel included in the playback environment information is the same as or higher than the channel included in the audio raw data, the audio data reproducing apparatus determines based on the analyzed audio metadata and preset playback environment information. Audio raw data can be reproduced. Also, the audio data reproducing apparatus may reproduce the audio raw data changed to the lower channel in operation 1130 .

Methods according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. A computer readable medium is any data storage device that can store data that can later be read by a computer system. Examples of computer-readable media include read-only memory, random access memory, CD-ROMs, DVDs, magnetic tapes, optical data storage devices, and carrier waves. The computer-readable medium may also be distributed across network-connected computer systems, such that the computer-readable code may be stored and executed in a distributed fashion. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited to the above embodiments, and various modifications and variations from these descriptions are provided by those skilled in the art to which the present invention pertains. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the following claims as well as the claims and equivalents.

110: audio data
120: audio raw data
130: audio metadata
140: audio signal

Claims (10)

generating information related to a channel of audio data;
providing the audio data and information related to the channel;
including,
The audio data is
An audio data providing method reproduced based on a channel identified according to a reproduction environment when the audio data is reproduced. According to claim 1,
Information related to the channel,
The method for providing audio data including name information of the channel, location information of the channel, and backward compatibility information of the channel. receiving audio data;
reproducing the audio data using information related to a channel of the audio data
including,
The audio data is
An audio data reproduction method that is reproduced based on a channel identified according to a reproduction environment when the audio data is reproduced. 4. The method of claim 3,
Information related to the channel,
An audio data reproduction method including name information of the channel, location information of the channel, and backward compatibility information of the channel. 4. The method of claim 3,
The channel-related information further includes information on the number of audio channels,
Information on the number of channels,
A method of playing audio data, which is the number of channels associated with the audio data. 4. The method of claim 3,
The location information of the channel,
An audio data reproduction method, which is information indicating where the channel should be spatially arranged. 4. The method of claim 3,
The location information of the channel,
A method of reproducing audio data including horizontal azimuth information and vertical azimuth information. 4. The method of claim 3,
The channel name information is
A method of reproducing audio data that is the name of each of the channels for the audio data. 5. The method of claim 4,
The backward compatibility information of the channel,
An audio data reproduction method which is information on a method of changing the audio data to a lower channel using a signal of a channel. identifying information related to a channel of audio data;
reproducing the audio data using information related to the channel
including,
The audio data is
An audio data reproduction method that is reproduced based on a channel identified according to a reproduction environment when the audio data is reproduced.

Images