KR20040060711A - Audio stream mixing method - Google Patents

Abstract

PURPOSE: A method for mixing audio streams is provided to mix different channel components without changing a channel structure of different audio streams, and to reproduce the mixed components, then to dynamically change the channel components, thereby supplying adaptive audio contents. CONSTITUTION: A reproduction device receives the first audio stream including plural audio data obtained from plural channels, respectively(901). The reproduction device receives mixing information from a network(902). The reproduction device decodes the first audio stream received from the network(903), and reads the second audio stream including plural audio data obtained from plural channels of a disk-type information storage medium(904). The reproduction device decodes the read second audio stream(905). The reproduction device mixes at least one of the audio data of the first audio stream with at least one of the audio data of the second audio stream(906).

Description

Audio stream mixing method

The present invention relates to a method of mixing audio data obtained from a plurality of channels, respectively.

1 is a schematic diagram of a user interface for adjusting the volume of an audio player installed in a conventional PC or the like.

The user may adjust the volume through the volume control interface as shown in FIG. 1. When the user adjusts the volume by raising or lowering the volume button 100 using the keyboard or the mouse, the audio mixing for simultaneously playing the audio data for each channel obtained from the plurality of channels simultaneously includes the number of channels of the audio stream to be reproduced, Regardless of the type of channel configured, this is done in any manner set by the player.

For example, when playing an audio stream composed of audio data obtained from two channels, the player decides in advance how much the output level of the audio data 1 of the channel 1 and the output level of the audio data 2 of the channel 2 is set in advance. It is specified. Accordingly, the audio data 1 and 2 of the channels 1 and 2 are mixed with each other by adjusting their output levels according to a predetermined size.

However, any mixing by the player can cause the following problems.

First, even if the content provider wants to simultaneously mix and adjust the output levels of audio data 1 and audio data 2 while providing the audio data 1 and 2 obtained from the two channels, respectively, they cannot be mixed according to the content provider's intention. . Because the adjustment factor for adjusting the output level is changed as preset in the player installed on the PC, it cannot properly reflect the intention of the content creator.

Second, once a method of mixing one audio content, such as a song or a dialogue for a movie, is determined, it is constantly mixed in that way until the end of playback. This makes it impossible to change the mix, thus providing more adaptive audio content.

Third, when channel components constituting the audio content are mixed with channel components of other audio content, only the same channel components may be mixed. In other words, even if the content provider wants to mix the audio data of the right channel of audio content 1 and the audio data of the left channel of audio content 2, this cannot be implemented. In particular, when one audio content is multi-channel data and the other audio content is two-channel data, it is currently impossible to mix the surround content of the multi-channel data without changing the channel structure of the two channel data. In order to mix the two channel data with one channel component of the multi channel data in the present method, the two channel data must be reconstructed into multi channel data, that is, the channel structure must be changed and transmitted. This requires resources to be used for data, resulting in waste of resources. In particular, if you play MP3 music downloaded from the Internet while playing a video with multi-channel audio attached, such as DVD-Video, the MP3 music is composed of two channels, the left channel and the right channel. Only the left and right channels of the channel audio are mixed and played back, and the size of the output level being mixed also depends on the characteristics of the player. Thus, even if the content provider wants to play MP3 music mixed with the size of the output level specified by the surround channel of the multi-channel audio of the DVD-Video, the content provider's intention cannot be realized in the present manner.

Accordingly, an aspect of the present invention is to provide a reproduction method in which different channel components are mixed and reproduced without changing the channel structure of an audio stream constituting different audio content.

Another technical problem to be solved by the present invention is to provide a playback method that can provide a more adaptive audio content by dynamically changing the mixing of a plurality of channel components in accordance with a change in content or characteristics of the content.

1 is a schematic diagram of a user interface for adjusting the volume of an audio player installed in a conventional PC, etc.

2A is a block diagram of a playback apparatus according to the present invention;

Figure 2b is an embodiment of Figure 2a,

3A and 3B are examples of an audio stream composed of a plurality of audio data respectively obtained from a plurality of channels according to the present invention;

4 is an embodiment of the playback device of FIG. 2A for mixing audio streams 1 and 2 of FIGS. 3A and 3B;

5 is a data structure diagram of mixing information according to the present invention;

FIG. 6 is an embodiment of a mixing table containing the mixing information of FIG. 5; FIG.

7 is a reference diagram for explaining dynamic mixing according to the present invention;

8 is a flowchart for explaining a reproducing method according to the present invention;

9 is a flowchart illustrating a reproducing method according to a preferred embodiment of the present invention;

10A and 10B show an implementation of step 906 of FIG. 9.

According to an aspect of the present invention, there is provided a method of reproducing an audio stream, the method comprising: (a) decoding an audio stream including a plurality of audio data respectively obtained from a plurality of channels; And (b) mixing the decoded plurality of audio data with reference to mixing information.

The step (b) includes adjusting the output level of the audio data to be mixed with reference to the mixing coefficient information of the mixing information, and referring to the mixing relationship information of the mixing information to display at least one of the plurality of audio data. It is preferable to include mixing so that the outputs are added to each other.

In addition, the technical problem according to the present invention comprises the steps of (a) decoding a first audio stream comprising a plurality of audio data obtained from a plurality of channels, respectively; (b) decoding a second audio stream comprising a plurality of audio data each obtained from the plurality of channels; And (c) mixing at least one audio data of the first audio stream and at least one audio data of the second audio stream with reference to mixing information.

Preferably, step (a) comprises receiving the first audio stream from a network.

(C) step (c1) finding audio data to which outputs should be added to each other according to channel information attached to the audio data with reference to the mixing relationship information; and (c2) outputting the found audio data to the mixing level; It is preferable to include adjusting according to the mixing coefficient information of the information.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

First, the meaning of "mixing" in the present invention is as follows. 1) adjusting the output level of audio data of at least some of the audio data of a plurality of channels constituting one audio stream, or 2) audio data of at least some of the audio data of a plurality of channels constituting one audio stream. Adjust the output level of and then add audio data of the adjusted output level to at least one of the remaining channels, or 3) add audio data of at least some of the audio data of a plurality of channels constituting one audio stream to each other, and then To export to. Alternatively, the above 1), 2), and 3) can be applied directly to at least some of the audio data of the plurality of channels constituting the plurality of audio streams, which is also a category of "mixing" in the present invention. Belongs to. Furthermore, the dynamic mixing described below also belongs to the mixing according to the present invention.

2A is a block diagram of a playback apparatus according to the present invention.

Referring to FIG. 2A, a reproduction device is a device for mixing at least one audio stream including a plurality of audio data obtained from a plurality of channels, respectively, based on mixing information according to the present invention, and includes a decoder 1 and a mixer 2. ). The decoder 1 decodes an audio stream including a plurality of audio data respectively obtained from the plurality of channels. The mixer 2 mixes a plurality of decoded audio data with reference to the mixing information. More specifically, the mixer 2 adjusts an output level of audio data of a mixing target channel with reference to mixing information, and outputs audio data of at least one channel of a plurality of channels constituting an audio stream with reference to the mixing information. Is added to the output of audio data of at least one channel constituting another audio stream. When there are a plurality of contents of the mixing information for one audio stream, the mixer 2 performs adaptive mixing without uniform mixing on one audio stream. Detailed description of the adaptive mixing will be given later.

FIG. 2B is an embodiment of FIG. 2A.

Referring to FIG. 2B, the playback apparatus includes a decoder 1, a mixer 2, a network transceiver 3, and a reader 4. The network transceiver 3 sends certain information to or receives information from the network. In particular, audio streams and / or mixing information are fetched from the network according to the invention. The reader 4 reads the audio stream and / or mixing information from a disc type information storage medium such as HD, CD, DVD. The audio stream includes a plurality of audio data respectively obtained from the plurality of channels. Mixing information is present in the network or stored in a disk type information storage medium. Detailed description of the mixing information will be described later.

The decoder 1 decodes the audio stream provided from the network transceiver 3 or the reader 4. The mixer 2 refers to the mixing information provided from the network transceiver 3 or the reader 4 to mix at least one audio data of the decoded first audio stream and at least one audio data of the second audio stream. More specifically, the mixer 2 adjusts the output level of the audio data of the mixing target channel with reference to the mixing information, and the last of the audio data of at least one channel among the plurality of channels constituting one audio stream with reference to the mixing information. The output is directed to the speaker in addition to the final output of the audio data of at least one of the plurality of channels constituting the other audio stream.

3A and 3B show an example of an audio stream composed of a plurality of audio data respectively obtained from a plurality of channels according to the present invention.

Audio stream 1 of FIG. 3A is an audio stream having five different channels and is composed of five audio data each obtained from five channels. The five channels consist of L (Left), R (Right), C (Center), Left Surround (LS), and RS (Right Surround). The L, R, and C channels provide a stable virtual sound source, while the LS and RS channels form a realistic 3D sound field. Audio data obtained from each channel according to the present invention has channel information. For example, if one audio data is obtained from an LS channel, the channel information indicates that the corresponding audio data is audio data of the LS channel.

Audio stream 2 of FIG. 3B is an audio stream having two channels and is composed of two audio data each obtained from the two channels. The two channels consist of L (Left) and R (Right). Two-channel audio stream can play the sound moving left and right. Similarly, audio data obtained from each channel according to the present invention has channel information. For example, if one audio data is obtained from the L channel, the channel information indicates that the audio data is audio data of the L channel.

4 is an embodiment of the playback device of FIG. 2A for mixing audio streams 1 and 2 of FIGS. 3A and 3B.

Referring to FIG. 4, the playback apparatus includes a decoder 1 having a decoder 1 11 and a decoder 2 12. The decoder 1 11 decodes five audio data corresponding to five channels constituting the audio stream 1, respectively, and outputs the audio data for each channel to the mixer 2. Decoder 2 12 decodes two pieces of audio data corresponding to two channels constituting audio stream 2 and outputs the respective pieces of audio data to the mixer 2.

The mixer 2 is composed of amplifiers 21, 22, 23, 24, 25, 26, 27 and at least two channels for amplifying the output levels of audio data output from decoder 1 11 and decoder 2 12. Adders 28 and 29 are provided for adding the output. However, the adder is shown only within the range necessary to describe the present embodiment, and of course, another adder may be further provided to add outputs of other channels not shown.

The mixer 2 multiplies the output levels of the L, R, and C channels output from the decoder 1 11 with the mixing coefficients 1 through the amplifiers 21, 22, and 23, respectively, with reference to the mixing information, and outputs the LS and RS channels. The level is multiplied by a mixing coefficient of 0.5 through amplifiers 24 and 25, respectively. Also, the mixer 2 multiplies the output levels of the L and R channels output from the decoder 2 12 with the mixing coefficients 0.5 through the amplifiers 26 and 27, respectively, with reference to the mixing information, and then adds them to the adder 28, respectively. 29) to the output of the audio data of the LS channel of the audio stream 1 and the output of the audio data of the RS channel of the audio stream 1 through. Accordingly, audio data of the L and R channels of the audio stream 2 is added to audio data of the LS and RS channels of the audio stream 1 and finally output as audio data of the five channels L, R, C, LS, and RS, respectively. .

5 is a data structure diagram of mixing information according to the present invention.

Referring to FIG. 5, the mixing information includes mixing relationship information and / or mixing coefficient information. Mixing relationship information indicates audio data to which outputs of the plurality of audio data are added to each other, and mixing coefficient information informs mixing coefficients for determining an output level of audio data to be mixed. In some embodiments, the mixing information may include only mixing relationship information or only mixing coefficient information.

FIG. 6 is an example of implementation of a mixing table that contains the mixing information of FIG. 5.

Referring to FIG. 6, the mixing table is for the mixer 2 of the playback apparatus of FIG. 4, and as the mixing relationship information, an identifier of an audio stream input to the mixer 2 and each audio stream input to the mixer 2. The channel component and the information about the object to be mixed, the identifier of the audio stream and the channel components to be added to each other are recorded, and the mixing coefficient for adjusting the output level is recorded as the mixing coefficient information. According to such a mixing table, the mixing coefficient 1 is multiplied by the output levels of the L, R, and C channels of the audio stream 1, and the mixing coefficient 0.5 is multiplied by the LS and RS channels. That is, the output levels of the LS and RS channels are cut in half and then added to the L and R channels of the audio stream 2, respectively. On the other hand, the output levels of the L and R channels of audio stream 2 are respectively multiplied by a mixing coefficient of 0.5. That is, the output levels of the L and R channels of the audio stream 2 are cut in half and then added to the LS and RS channels of the audio stream 1, respectively.

For example, if audio stream 1 is an AC3 stream and audio stream 2 is an MP3 stream, mixer 2 cuts the output level of the LS and RS channels of the AC3 stream in half according to the mixing table, and the L and R channels of the MP3 stream. Cut the output level in half and add them to each other for final export to the LS and RS channels.

7 is a reference diagram for explaining dynamic mixing according to the present invention.

Referring to FIG. 7, when there is an audio stream composed of L and R channels attached to a video and played with the video, for example, when a movie is played, a narration explaining the intention of the movie director is played together, Playing audio streams with fixed mixing coefficients from start to finish can be inappropriate. In other words, if the director's narration is played at the same output level in a scene that emphasizes a quiet atmosphere, or in a noisy battle scene, the output level of the narration may be so large that it may not match the mood of the movie in a scene that emphasizes a quiet atmosphere. In narration, the output level of the narration is too small to be heard. Accordingly, the content provider may create and provide a plurality of mixing tables in which mixing coefficients capable of adjusting the output level according to the movie are recorded. When a plurality of mixing tables are provided, reference timing information is attached. The reference timing information is information for informing the mixer 2 of the playback apparatus when to refer to the mixing table. Accordingly, the mixer 2 may adjust the output level by multiplying different mixing coefficients by referring to the plurality of mixing tables, respectively, according to the reference timing information. Thus, the mixing in the present invention includes dynamic mixing.

The reproduction method according to the preferred embodiment of the present invention based on the above configuration will be described.

8 is a flowchart for explaining a reproducing method according to the present invention.

Referring to FIG. 8, the playback apparatus decodes an audio stream including a plurality of audio data respectively obtained from a plurality of channels (step 801), and then mixes at least some of the decoded plurality of audio data with reference to mixing information. (Step 802). The plurality of audio data may belong to one audio stream or at least some may belong to different audio streams.

9 is a flowchart for explaining a playback method according to a preferred embodiment of the present invention.

Referring to FIG. 9, the reproduction apparatus receives a first audio stream including a plurality of audio data respectively obtained from a plurality of channels from a network (step 901). In addition, the mixing information is received from the network (step 902). Next, while decoding the first audio stream received from the network (step 903), a second audio stream including a plurality of audio data respectively obtained from the plurality of channels is read from the disc type information storage medium (step 904). Decode it (step 905). Finally, the reproduction apparatus mixes at least one audio data of the first audio stream and at least one audio data of the second audio stream with reference to the received mixing information (step 906).

10A and 10B show implementations of step 906 of FIG. 9.

Referring to FIG. 10A, the reproduction apparatus adjusts an output level of audio data to be mixed with reference to mixing coefficient information of the mixing information, and then adds at least one of the plurality of audio data to each other by referring to mixing relationship information of the mixing information. Mix to lose (step 1001).

Referring to FIG. 10B, the playback apparatus finds audio data to which outputs should be added to each other according to channel information attached to audio data with reference to mixing relationship information, and adjusts and outputs the output levels of the found audio data according to mixing coefficient information. (Step 1002).

As described above, according to the present invention, different channel components can be mixed and reproduced without changing the channel structure of different audio streams. In addition, the mixing of the plurality of channel components may be dynamically changed in accordance with the change in the content or characteristics of the content, thereby providing more adaptive audio content.

Claims (9)

In the method of playing an audio stream, (a) decoding an audio stream comprising a plurality of audio data respectively obtained from the plurality of channels; And (b) mixing at least a portion of the decoded plurality of audio data with reference to mixing information. The method of claim 1, Step (b) is And adjusting and mixing an output level of audio data with reference to mixing coefficient information of the mixing information. The method of claim 1, Step (b) is And mixing at least one output of the plurality of audio data to be added to each other with reference to the mixing relationship information of the mixing information. (a) decoding a first audio stream comprising a plurality of audio data respectively obtained from the plurality of channels; (b) decoding a second audio stream comprising a plurality of audio data each obtained from the plurality of channels; And (c) mixing at least one audio data of the first audio stream and at least one audio data of the second audio stream with reference to mixing information. The method of claim 4, wherein Step (a) is Receiving the first audio stream from a network. The method of claim 4, wherein Step (c) is Receiving mixing information from the network. The method of claim 4, wherein Step (c) is And adjusting and mixing an output level of audio data to be mixed with reference to the mixing coefficient information of the mixing information. The method of claim 4, wherein Step (c) is And mixing at least one output of the plurality of audio data to be added to each other with reference to the mixing relationship information of the mixing information. The method of claim 4, wherein Step (c) is (c1) finding audio data to which outputs should be added to each other according to channel information attached to the audio data with reference to the mixing relationship information; And (c2) adjusting the output level of the found audio data according to the mixing coefficient information of the mixing information.