JP2023072027A - Decoder and method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the amount of calculation for decoding an audio signal to be reduced.

SOLUTION: A priority information acquisition unit acquires priority information of each channel from a bit stream for supply to an output selection unit. A channel audio signal decoding unit decodes encoded data of an audio signal of each channel, and supplies an obtained MDCT coefficient to the output selection unit. The output selection unit supplies the MDCT coefficient to an IMDCT unit in the case where the priority degree indicated in the priority information is more than or equal to a predetermined degree, and supplies 0 as the MDCT coefficient to a 0-value output unit in the case where the priority degree is less than the predetermined degree. The 0-value output unit generates an audio signal on the basis of 0 supplied as the MDCT coefficient. In addition, the IMDCT unit performs IMDCT on the basis of the MDCT coefficient to generate an audio signal. The present technology can be applied to an encoder and a decoder.

SELECTED DRAWING: Figure 10

COPYRIGHT: (C)2023,JPO&INPIT

Description

TECHNICAL FIELD The present technology relates to an encoding device and method, a decoding device and method, and a program, and more particularly to an encoding device and method, a decoding device and method, and a program that can reduce the amount of calculation for decoding an audio signal. .

For example, the MPEG (Moving Picture Experts Group)-2 AAC (Advanced Audio Coding) standard, the MPEG-4 AAC standard, and the MPEG-D USAC (Unified Speech and Audio Coding) standard, which are international standards, are used as methods for encoding audio signals. Standard multi-channel coding is known (see, for example, Non-Patent Document 1 and Non-Patent Document 2).

INTERNATIONAL STANDARD ISO/IEC 14496-3 Fourth edition 2009-09-01 Information technology-coding of audio-visual objects-part3:Audio INTERNATIONAL STANDARD ISO/IEC 23003-3 Frist edition 2012-04-01 Information technology-coding of audio-visual objects-part3: Unified speech and audio coding

By the way, encoding technology using more audio channels is required for playback with a higher sense of reality than conventional 5.1-channel surround playback and for transmission of multiple sound materials (objects).

For example, consider a case of encoding and decoding a 24-channel audio signal and audio signals of a plurality of objects, and a case of encoding and decoding a 2-channel audio signal. In such a case, it is possible to decode a 2-channel audio signal in real time on a mobile device with poor computing power, but it is not possible to decode a 24-channel audio signal and multiple object audio signals in real time. It can be difficult.

Since current audio codecs such as MPEG-D USAC need to decode audio signals of all channels and all objects, it is difficult to reduce the amount of computation during decoding. As a result, depending on the device on the decoding side, it may become impossible to reproduce the audio signal in real time.

The present technology has been made in view of such circumstances, and is intended to reduce the computational complexity of decoding.

A decoding device according to a first aspect of the present technology includes an acquisition unit that acquires encoded audio signals of a plurality of channels or a plurality of objects and priority information of each of the audio signals at a predetermined time; an audio signal decoding unit for decoding, based on the information, the encoded audio signal of a predetermined number of channels or objects according to the priority information.

A decoding method or program according to the first aspect of the present technology obtains encoded audio signals of a plurality of channels or a plurality of objects and priority information of each of the audio signals at a predetermined time, and obtains the priority information decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information, based on.

In a first aspect of the present technology, encoded audio signals of multiple channels or multiple objects and priority information of each of the audio signals at a predetermined time are obtained, and based on the priority information, The encoded audio signal of a predetermined number of channels or objects according to the priority information is decoded.

A coding apparatus according to a second aspect of the present technology includes a priority information generation unit that generates priority information at a predetermined time of audio signals of multiple channels or multiple objects, and stores the priority information in a bitstream. and a packing part.

A coding method or program according to a second aspect of the present technology includes generating priority information at a predetermined time of an audio signal of multiple channels or multiple objects, and storing the priority information in a bitstream. .

In a second aspect of the present technology, priority information at a given time of an audio signal of multiple channels or multiple objects is generated, and the priority information is stored in a bitstream.

FIG. 3 is a diagram for explaining a bitstream; FIG. It is a figure explaining encoding. It is a figure explaining priority information. FIG. 4 is a diagram for explaining the meaning of values of priority information; It is a figure which shows the structural example of an encoding apparatus. FIG. 4 is a diagram showing a configuration example of a channel audio encoding unit; FIG. 3 is a diagram showing a configuration example of an object audio encoding unit; 4 is a flowchart for explaining encoding processing; It is a figure which shows the structural example of a decoding apparatus. FIG. 4 is a diagram showing a configuration example of an unpacking/decoding unit; 4 is a flowchart for explaining decoding processing; 10 is a flowchart for explaining selective decoding processing; FIG. 10 is a diagram showing another configuration example of the unpacking/decoding unit; 10 is a flowchart for explaining selective decoding processing; FIG. 4 is a diagram illustrating an example of syntax of object metadata; FIG. 4 is a diagram explaining generation of an audio signal; FIG. 4 is a diagram explaining generation of an audio signal; FIG. 10 is a diagram illustrating selection of an output destination of MDCT coefficients; It is a figure explaining the gain adjustment of the power value of an audio signal and a high frequency band. It is a figure explaining the gain adjustment of the power value of an audio signal and a high frequency band. FIG. 10 is a diagram showing another configuration example of the unpacking/decoding unit; 10 is a flowchart for explaining selective decoding processing; It is a figure explaining the gain adjustment of an audio signal. It is a figure explaining the gain adjustment of an audio signal. FIG. 10 is a diagram showing another configuration example of the unpacking/decoding unit; 10 is a flowchart for explaining selective decoding processing; FIG. 4 is a diagram for explaining VBAP gain; FIG. 4 is a diagram for explaining VBAP gain; FIG. 10 is a diagram showing another configuration example of the unpacking/decoding unit; 4 is a flowchart for explaining decoding processing; 10 is a flowchart for explaining selective decoding processing; It is a figure which shows the structural example of a computer.

Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

<First Embodiment>
<Outline of this technology>
This technology transmits priority information of audio signals of each channel and priority information of audio signals of each object in encoding of audio signals of each channel and audio signals of objects that constitute a multi-channel. This is intended to reduce the computational complexity of decoding.

In addition, the present technology performs frequency-time conversion on the decoding side when the priority indicated in the priority information of each channel or each object is equal to or higher than a predetermined degree, and the priority indicated in the priority information reaches the predetermined degree. If it is less than that, frequency-time conversion is not performed, and the result of frequency-time conversion is set to 0, thereby reducing the amount of computation for decoding the audio signal.

In the following description, the case where the multi-channel audio signal and the object audio signal are encoded according to the AAC standard will be described, but similar processing is performed when they are encoded by other methods.

For example, when multi-channel audio signals and audio signals of a plurality of objects are encoded according to the AAC standard and transmitted, the audio signals of each channel and each object are encoded and transmitted frame by frame.

Specifically, as shown in FIG. 1, encoded audio signals and information necessary for decoding audio signals are stored in a plurality of elements (bitstream elements), and a bitstream consisting of these elements is transmitted. will be

In this example, a bit stream for one frame has t elements EL1 to ELt arranged in order from the beginning, and finally an identifier TERM indicating the end position of the information of the frame. .

For example, the element EL1 placed at the head is an ancillary data area called a DSE (Data Stream Element), and the DSE describes information on each of a plurality of channels, such as information on downmixing of audio signals and identification information. be.

Encoded audio signals are stored in the elements EL2 to ELt following the element EL1.

In particular, an element storing a single-channel audio signal is called an SCE, and an element storing a pair of two-channel audio signals is called a CPE. Also, the audio signal of each object is stored in the SCE.

In this technology, the priority information of the audio signal of each channel constituting the multi-channel and the priority information of the audio signal of each object are generated and stored in the DSE.

For example, as shown in FIG. 2, it is assumed that an audio signal of consecutive frames F11 to F13 is encoded.

In such a case, an encoding device (encoder) analyzes the degree of priority of the audio signal of each channel for each of those frames, and for example, the priority information of each channel as shown in FIG. to generate Similarly, the encoding device also generates priority information for the audio signal of each object.

For example, the encoding device analyzes the degree of priority of the audio signal based on the sound pressure and spectral shape of the audio signal, the correlation of spectral shape between channels and between objects, and the like.

In FIG. 3, priority information of each channel is shown as an example when the total number of channels is M channels. That is, for each channel from channel number 0 to channel number M−1, numerical values indicating the priority levels of the audio signals of these channels are indicated as priority information.

For example, the priority information of a channel with a channel number of 0 is 3, and the priority information of a channel with a channel number of 1 is 0. In addition, hereinafter, a channel with a predetermined channel number m (m=0, 1, . . . , M-1) is also referred to as channel m.

The value of the priority information shown in FIG. 3 is set to any value from 0 to 7 as shown in FIG. is considered to be of high priority, that is, of high importance.

Therefore, an audio signal with a priority information value of 0 has the lowest priority, and an audio signal with a priority information value of 7 has the highest priority.

When multi-channel audio signals or audio signals of multiple objects are played simultaneously, some of the sounds reproduced by those audio signals usually contain less important sounds than other sounds. In other words, even if a particular sound is not reproduced in the entire sound, there are sounds that do not cause the listener to feel uncomfortable.

Therefore, if audio signals with low priority are not decoded as necessary, it is possible to reduce the computational complexity of decoding while suppressing deterioration in sound quality. Therefore, in the encoding apparatus, priority information indicating the degree of importance of each audio signal during reproduction, that is, the degree of priority for decoding, is stored for each frame so that the audio signal that is not to be decoded can be appropriately selected. is assigned to each audio signal.

When the priority information of each audio signal is determined as described above, the priority information is stored in the DSE of the element EL1 shown in FIG. Especially in the example of FIG. 3, since the number of channels constituting the multi-channel audio signal is M, the priority information of each of M channels from channel 0 to channel M-1 is stored in the DSE.

Similarly, priority information for each object is also stored in the DSE of element EL1. Here, for example, if there are N objects with object numbers from 0 to N-1, priority information is determined for each of the N objects and stored in the DSE.

An object with a predetermined object number n (n=0, 1, . . . , N-1) is hereinafter also referred to as an object n.

In this way, if priority information is defined for each audio signal, the playback side, that is, the audio signal decoding side, determines which audio signal is important during playback and should be decoded preferentially. You can easily identify what to do.

Returning to the description of FIG. 2, for example, it is assumed that the priority information of the audio signals of the frames F11 and F13 of the predetermined channel is 7, and the priority information of the audio signal of the frame F12 of the predetermined channel is 0.

In addition, it is assumed that the audio signal decoding side, ie, the decoding device (decoder), does not decode audio signals having a priority lower than a predetermined degree.

Here, for example, the predetermined priority is called a threshold, and if the threshold is 4, in the above example, the audio signals of the frames F11 and F13 of the predetermined channel whose priority information is 7 are is decrypted.

On the other hand, the audio signal of the frame F12 of the predetermined channel whose priority information is 0 is not decoded.

Therefore, in this example, the audio signal of the frame F12 is treated as a silent signal, and the audio signals of the frames F11 and F13 are combined to obtain the final audio signal of the predetermined channel.

More specifically, for example, when each audio signal is encoded, time-frequency transform is performed on the audio signal, the information obtained by the time-frequency transform is encoded, and the encoded data obtained as a result is stored in the element. be.

Any processing may be performed as the time-frequency transform, but the following description will continue assuming that MDCT (Modified Discrete Cosine Transform) is performed as the time-frequency transform.

In addition, the decoding device decodes the encoded data, performs IMDCT (Inverse Modified Discrete Cosine Transform) on the resulting MDCT coefficients, and generates an audio signal. . That is, here, IMDCT is performed as inverse transform (frequency-time transform) of time-frequency transform.

Therefore, more specifically, IMDCT is performed on the frames F11 and F13 whose priority information is equal to or greater than the threshold value of 4, and audio signals are generated.

Also, the IMDCT is not performed for the frame F12 whose priority information is less than the threshold value of 4, and the result of the IMDCT is set to 0 and an audio signal is generated. As a result, the audio signal of frame F12 becomes a silent signal, that is, 0 data.

As another example, in the example shown in FIG. 3, when the threshold is 4, among the audio signals of each channel 0 to channel M−1, the priority information is less than the threshold value of 4. 0, channel 1, and channel M-2 audio signals will not be decoded.

As described above, according to the comparison result with the threshold, decoding is not performed for audio signals with a low priority indicated by the priority information. amount can be reduced.

<Configuration example of encoding device>
Next, specific embodiments of an encoding device and a decoding device to which the present technology is applied will be described. First, the encoding device will be explained.

FIG. 5 is a diagram showing a configuration example of an encoding device to which the present technology is applied.

The encoding device 11 of FIG. 5 has a channel audio encoding section 21, an object audio encoding section 22, a metadata input section 23, and a packing section 24.

Audio signals of each channel of a multi-channel having M channels are supplied to the channel audio encoding unit 21 . For example, the audio signals for each channel are supplied from microphones corresponding to those channels. In FIG. 5, the characters "#0" to "#M-1" represent the channel number of each channel.

The channel audio encoding unit 21 encodes the supplied audio signal of each channel, generates priority information based on the audio signal, and encodes the encoded data obtained by encoding and the priority information. It is supplied to the packing section 24 .

Audio signals of each of the N objects are supplied to the object audio encoding unit 22 . For example, the audio signal for each object is provided by microphones attached to those objects. In FIG. 5, the characters "#0" to "#N-1" represent the object number of each object.

The object audio encoding unit 22 encodes the supplied audio signal of each object, generates priority information based on the audio signal, and encodes the encoded data obtained by encoding and the priority information. It is supplied to the packing section 24 .

The metadata input section 23 supplies the metadata of each object to the packing section 24 . For example, metadata of an object is spatial position information indicating the position of the object in space. More specifically, for example, the spatial position information is three-dimensional coordinate information indicating the coordinates of the position of the object in the three-dimensional space.

The packing unit 24 stores the encoded data and priority information supplied from the channel audio encoding unit 21, the encoded data and priority information supplied from the object audio encoding unit 22, and the metadata input unit 23. Generate and output a bitstream by packing the metadata.

The bitstream thus obtained includes encoded data for each channel, priority information for each channel, encoded data for each object, priority information for each object, and metadata for each object for each frame. It means that

Here, the audio signals of the M channels and the audio signals of the N objects stored in the bitstream for one frame are audio signals of the same frame to be reproduced simultaneously.

Here, as the priority information of the audio signal of each channel and each object, an example will be described in which priority information is generated for each audio signal for each frame. , for example, one piece of priority information may be generated for several frames of audio signals.

<Configuration example of channel audio encoder>
Further, the channel audio encoding unit 21 in FIG. 5 is more specifically configured as shown in FIG. 6, for example.

The channel audio encoder 21 shown in FIG. 6 includes an encoder 51 and a priority information generator 52 .

The encoding unit 51 includes an MDCT unit 61, and the encoding unit 51 encodes the audio signal of each channel supplied from the outside.

That is, the MDCT section 61 performs MDCT on the externally supplied audio signal of each channel. The encoding unit 51 encodes the MDCT coefficients of each channel obtained by the MDCT, and supplies the resulting encoded data of each channel, that is, the encoded audio signal to the packing unit 24 .

The priority information generator 52 also analyzes the externally supplied audio signal of each channel, generates priority information of the audio signal of each channel, and supplies it to the packing unit 24 .

<Configuration example of object audio encoding unit>
Further, the object audio encoding unit 22 of FIG. 5 is configured, more specifically, as shown in FIG. 7, for example.

The object audio encoding section 22 shown in FIG. 7 includes an encoding section 91 and a priority information generating section 92 .

The encoding unit 91 includes an MDCT unit 101, and the encoding unit 91 encodes the audio signal of each object supplied from the outside.

That is, the MDCT unit 101 performs MDCT on the audio signal of each object supplied from the outside. The encoding unit 91 encodes the MDCT coefficients of each object obtained by MDCT, and supplies the resulting encoded data of each object, that is, the encoded audio signal to the packing unit 24 .

The priority information generation unit 92 also analyzes the audio signal of each object supplied from the outside, generates priority information of the audio signal of each object, and supplies the information to the packing unit 24 .

<Description of encoding processing>
Next, processing performed by the encoding device 11 will be described.

When the audio signals of each of the plurality of channels and the audio signals of each of the plurality of objects, which are reproduced simultaneously, are supplied for only one frame, the encoding device 11 performs encoding processing to generate encoded audio signals. Output the contained bitstream.

The encoding process by the encoding device 11 will be described below with reference to the flowchart of FIG. Note that this encoding process is performed for each frame of the audio signal.

In step S<b>11 , the priority information generation unit 52 of the channel audio encoding unit 21 generates priority information of the supplied audio signal of each channel and supplies it to the packing unit 24 . For example, the priority information generation unit 52 analyzes the audio signal for each channel and generates priority information based on the sound pressure and spectrum shape of the audio signal, the correlation of the spectrum shape between channels, and the like.

In step S12, the packing unit 24 stores the priority information of the audio signal of each channel supplied from the priority information generating unit 52 in the DSE of the bitstream. That is, priority information is stored in the leading element of the bitstream.

In step S<b>13 , the priority information generation unit 92 of the object audio encoding unit 22 generates priority information of the supplied audio signal of each object and supplies it to the packing unit 24 . For example, the priority information generator 92 analyzes the audio signal for each object and generates priority information based on the sound pressure and spectral shape of the audio signal, the correlation of spectral shapes between objects, and the like.

When generating priority information for audio signals of each channel and each object, the number of audio signals to which each priority, which is the value of the priority information, is assigned is determined by the number of channels and the number of objects. may be determined in advance.

For example, in the example of FIG. 3, the number of audio signals whose priority information is "7", that is, the number of channels is five, and the number of audio signals whose priority information is "6" is three. It may be determined in advance.

In step S14, the packing unit 24 stores the priority information of the audio signal of each object supplied from the priority information generation unit 92 in the DSE of the bitstream.

In step S15, the packing unit 24 stores the metadata of each object in the DSE of the bitstream.

For example, the metadata input unit 23 acquires the metadata of each object by receiving user input operations, communicating with the outside, or reading data from an external recording area. supply to The packing unit 24 stores the metadata supplied from the metadata input unit 23 in this way in the DSE.

By the above processing, the DSE of the bitstream stores the priority information of audio signals of all channels, the priority information of audio signals of all objects, and the metadata of all objects.

In step S16, the encoding unit 51 of the channel audio encoding unit 21 encodes the supplied audio signal of each channel.

More specifically, the MDCT unit 61 performs MDCT on the audio signal of each channel, the encoding unit 51 encodes the MDCT coefficients of each channel obtained by the MDCT, and the resulting The encoded data is supplied to the packing section 24 .

In step S17, the packing unit 24 stores the encoded data of the audio signal of each channel supplied from the encoding unit 51 in the SCE or CPE of the bitstream. In other words, coded data is stored in each element arranged following the DSE in the bitstream.

In step S18, the encoding unit 91 of the object audio encoding unit 22 encodes the supplied audio signal of each object.

More specifically, MDCT section 101 performs MDCT on the audio signal of each object, encoding section 91 encodes the MDCT coefficients of each object obtained by MDCT, and encodes the resulting The encoded data is supplied to the packing section 24 .

In step S19, the packing unit 24 stores the encoded data of the audio signal of each object supplied from the encoding unit 91 in the SCE of the bitstream. That is, encoded data is stored in some elements located after the DSE in the bitstream.

With the above processing, the priority information and encoded data of audio signals for all channels, the priority information and encoded data of audio signals for all objects, and the metadata of all objects are stored for the frame being processed. resulting in a compressed bitstream.

In step S20, the packing unit 24 outputs the obtained bitstream, and the encoding process ends.

As described above, the encoding device 11 generates the priority information of the audio signal of each channel and the priority information of the audio signal of each object, stores them in a bitstream, and outputs them. Therefore, the decoding side can easily grasp which audio signal has a higher priority.

This allows the decoding side to selectively decode the encoded audio signal according to the priority information. As a result, it is possible to reduce the computational complexity of decoding while minimizing the deterioration of the sound quality of the sound reproduced by the audio signal.

In particular, by storing the priority information of the audio signal of each object in the bitstream, it is possible to not only reduce the computational complexity of decoding on the decoding side, but also reduce the computational complexity of subsequent processing such as rendering. can.

<Configuration example of decoding device>
Next, a decoding device that receives as input the bitstream output from the encoding device 11 described above and decodes the encoded data contained in the bitstream will be described.

Such a decoding device is configured, for example, as shown in FIG.

Decoding device 151 shown in FIG. 9 has unpacking/decoding section 161 , rendering section 162 and mixing section 163 .

The unpacking/decoding unit 161 acquires the bitstream output from the encoding device 11, and unpacks and decodes the bitstream.

The unpacking/decoding unit 161 supplies the audio signal of each object obtained by unpacking and decoding and the metadata of each object to the rendering unit 162 . At this time, the unpacking/decoding unit 161 decodes the encoded data of each object according to the priority information included in the bitstream.

The unpacking/decoding section 161 also supplies the audio signal of each channel obtained by unpacking and decoding to the mixing section 163 . At this time, the unpacking/decoding unit 161 decodes the encoded data of each channel according to the priority information included in the bitstream.

The rendering unit 162 generates an M-channel audio signal based on the audio signal of each object supplied from the unpacking/decoding unit 161 and the spatial position information as metadata of each object, and supplies the M-channel audio signal to the mixing unit 163. do. At this time, the rendering unit 162 generates audio signals for each of the M channels so that the sound image of each object is localized at the position indicated by the spatial position information of those objects.

The mixing unit 163 performs weighted addition for each channel on the audio signal of each channel supplied from the unpacking/decoding unit 161 and the audio signal of each channel supplied from the rendering unit 162, and finalizes each channel. to generate an audio signal. The mixing unit 163 supplies the final audio signals of each channel thus obtained to the external speakers corresponding to each channel to reproduce the sound.

<Configuration example of unpacking/decoding section>
Further, the unpacking/decoding section 161 of the decoding device 151 shown in FIG. 9 is more specifically configured as shown in FIG. 10, for example.

The unpacking/decoding unit 161 shown in FIG. 10 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, an object audio It has a signal acquisition section 197 , an object audio signal decoding section 198 , an output selection section 199 , a 0 value output section 200 and an IMDCT section 201 .

The priority information acquisition unit 191 acquires the priority information of the audio signal of each channel from the supplied bitstream and supplies it to the output selection unit 194, and also acquires the priority information of the audio signal of each object from the bitstream. It is acquired and supplied to the output selection unit 199 .

Also, the priority information acquisition unit 191 acquires metadata of each object from the supplied bitstream and supplies it to the rendering unit 162, and supplies the bitstream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197. supply.

The channel audio signal acquisition unit 192 acquires encoded data of each channel from the bitstream supplied from the priority information acquisition unit 191 and supplies the encoded data to the channel audio signal decoding unit 193 . The channel audio signal decoding unit 193 decodes the encoded data of each channel supplied from the channel audio signal acquisition unit 192 and supplies the resulting MDCT coefficients to the output selection unit 194 .

The output selection unit 194 selectively switches the output destination of the MDCT coefficients of each channel supplied from the channel audio signal decoding unit 193 based on the priority information of each channel supplied from the priority information acquisition unit 191 .

That is, when the priority information for a given channel is less than a given threshold value P, the output selection section 194 sets the MDCT coefficient of that channel to 0 and supplies it to the 0 value output section 195 . Also, when the priority information for a given channel is equal to or greater than a given threshold value P, the output selection section 194 supplies the MDCT coefficients of that channel supplied from the channel audio signal decoding section 193 to the IMDCT section 196 .

The 0-value output unit 195 generates an audio signal based on the MDCT coefficients supplied from the output selection unit 194 and supplies the audio signal to the mixing unit 163 . In this case, since the MDCT coefficient is 0, a silent audio signal is generated.

The IMDCT section 196 performs IMDCT based on the MDCT coefficients supplied from the output selection section 194 to generate an audio signal, and supplies the audio signal to the mixing section 163 .

The object audio signal acquisition unit 197 acquires encoded data of each object from the bitstream supplied from the priority information acquisition unit 191 and supplies the encoded data to the object audio signal decoding unit 198 . The object audio signal decoding unit 198 decodes the encoded data of each object supplied from the object audio signal acquisition unit 197 and supplies the resulting MDCT coefficients to the output selection unit 199 .

The output selection unit 199 selectively switches the output destination of the MDCT coefficients of each object supplied from the object audio signal decoding unit 198 based on the priority information of each object supplied from the priority information acquisition unit 191 .

That is, when the priority information for a given object is less than a given threshold Q, the output selector 199 sets the MDCT coefficient of that object to 0 and supplies it to the 0 value output section 200 . Also, when the priority information for a given object is equal to or greater than a given threshold Q, the output selector 199 supplies the MDCT coefficients of the object supplied from the object audio signal decoder 198 to the IMDCT unit 201 .

Note that the value of the threshold Q may be the same as the value of the threshold P, or may be a value different from the value of the threshold P. By appropriately determining the threshold value P and the threshold value Q according to the computational capacity of the decoding device 151, the computational complexity of decoding the audio signal is reduced to the computational complexity within the range that the decoding device 151 can decode in real time. can be made

The 0-value output unit 200 generates an audio signal based on the MDCT coefficients supplied from the output selection unit 199 and supplies the audio signal to the rendering unit 162 . In this case, since the MDCT coefficient is 0, a silent audio signal is generated.

The IMDCT unit 201 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 199 to generate an audio signal, and supplies the audio signal to the rendering unit 162 .

<Description of Decryption Processing>
Next, the operation of the decoding device 151 will be described.

When the bit stream for one frame is supplied from the encoding device 11, the decoding device 151 performs decoding processing to generate an audio signal and outputs the audio signal to a speaker. The decoding process performed by the decoding device 151 will be described below with reference to the flowchart of FIG.

In step S<b>51 , the unpacking/decoding unit 161 acquires the bitstream transmitted from the encoding device 11 . That is, a bitstream is received.

In step S52, the unpacking/decoding unit 161 performs selective decoding processing.

Although details of the selective decoding process will be described later, in the selective decoding process, encoded data of each channel and encoded data of each object are selectively decoded based on priority information. Then, the audio signal of each channel obtained as a result is supplied to the mixing unit 163 , and the audio signal of each object is supplied to the rendering unit 162 . Metadata of each object obtained from the bitstream is also supplied to the rendering unit 162 .

In step S53, the rendering unit 162 renders the audio signal of each object based on the audio signal of each object supplied from the unpacking/decoding unit 161 and the spatial position information as metadata of each object.

For example, the rendering unit 162 generates audio signals for each channel by VBAP (Vector Base Amplitude Panning) based on the spatial position information so that the sound image of each object is localized at the position indicated by the spatial position information, and the mixing unit 163 supply to

In step S54, the mixing unit 163 weights and adds the audio signal of each channel supplied from the unpacking/decoding unit 161 and the audio signal of each channel supplied from the rendering unit 162 for each channel. supply to As a result, each speaker is supplied with the audio signal of the channel corresponding to the speaker, so that each speaker reproduces sound based on the supplied audio signal.

The decoding process ends when the audio signal of each channel is supplied to the speaker.

As described above, the decoding device 151 acquires priority information from the bitstream and decodes the encoded data of each channel and each object according to the priority information.

<Description of selective decryption processing>
Next, the selective decoding process corresponding to the process of step S52 in FIG. 11 will be described with reference to the flowchart in FIG.

In step S81, the priority information acquisition unit 191 acquires the priority information of the audio signal of each channel and the priority information of the audio signal of each object from the supplied bitstream, and selects the output selection unit 194 and the output selection unit 194 respectively. It is supplied to the output selection section 199 .

The priority information acquisition unit 191 also acquires metadata of each object from the bitstream and supplies the metadata to the rendering unit 162 , and supplies the bitstream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197 .

In step S82, the channel audio signal acquisition unit 192 sets 0 to the channel number of the channel to be processed and holds it.

In step S83, the channel audio signal acquisition unit 192 determines whether or not the retained channel number is less than the number M of channels.

If it is determined in step S83 that the channel number is less than M, in step S84 the channel audio signal decoding unit 193 decodes the encoded data of the audio signal of the channel to be processed.

That is, the channel audio signal acquisition unit 192 acquires encoded data of the channel to be processed from the bitstream supplied from the priority information acquisition unit 191 and supplies the encoded data to the channel audio signal decoding unit 193 .

Then, the channel audio signal decoding unit 193 decodes the encoded data supplied from the channel audio signal acquisition unit 192 and supplies the resulting MDCT coefficients to the output selection unit 194 .

In step S85, the output selection unit 194 determines whether the priority information of the channel to be processed supplied from the priority information acquisition unit 191 is equal to or greater than a threshold value P designated by an unillustrated host control device or the like. judge. Here, the threshold value P is determined according to the computing power of the decoding device 151, for example.

In step S85, when it is determined that the priority information is equal to or greater than the threshold value P, the output selection unit 194 supplies the MDCT coefficients of the channel to be processed, supplied from the channel audio signal decoding unit 193, to the IMDCT unit 196. , the process proceeds to step S86. In this case, since the priority of the audio signal of the channel to be processed is equal to or higher than the predetermined priority, decoding, more specifically IMDCT, is performed on that channel.

In step S<b>86 , the IMDCT section 196 performs IMDCT based on the MDCT coefficients supplied from the output selection section 194 to generate an audio signal of the channel to be processed, and supplies the audio signal to the mixing section 163 . After the audio signal is generated, the process proceeds to step S87.

On the other hand, when it is determined in step S85 that the priority information is less than the threshold value P, the output selection unit 194 sets the MDCT coefficient to 0 and supplies it to the 0 value output unit 195 .

The 0-value output unit 195 generates an audio signal of the channel to be processed from the MDCT coefficients of 0 supplied from the output selection unit 194 , and supplies the audio signal to the mixing unit 163 . Therefore, the 0-value output unit 195 does not substantially perform any processing for generating an audio signal such as IMDCT.

Note that the audio signal generated by the 0-value output unit 195 is a silent signal. After the audio signal is generated, the process proceeds to step S87.

If it is determined in step S85 that the priority information is less than the threshold value P, or if an audio signal is generated in step S86, then in step S87 the channel audio signal acquisition unit 192 adds 1 to the retained channel number. In addition, update the channel number of the channel to be processed.

After the channel number is updated, the process returns to step S83 and the above-described processes are repeated. That is, the audio signal of the new channel to be processed is generated.

Further, when it is determined in step S83 that the channel number of the channel to be processed is not less than M, the process proceeds to step S88 because audio signals have been obtained for all channels.

In step S88, the object audio signal acquisition unit 197 sets 0 to the object number of the object to be processed and holds it.

In step S89, the object audio signal acquisition unit 197 determines whether or not the retained object number is less than the number N of objects.

If it is determined in step S89 that the object number is less than N, in step S90 the object audio signal decoding unit 198 decodes the encoded data of the audio signal of the object to be processed.

That is, the object audio signal acquisition unit 197 acquires encoded data of the object to be processed from the bitstream supplied from the priority information acquisition unit 191 and supplies the encoded data to the object audio signal decoding unit 198 .

Then, the object audio signal decoding unit 198 decodes the encoded data supplied from the object audio signal acquisition unit 197 and supplies the resulting MDCT coefficients to the output selection unit 199 .

In step S91, the output selection unit 199 determines whether the priority information of the object to be processed, which is supplied from the priority information acquisition unit 191, is equal to or greater than a threshold value Q specified by a higher control device (not shown). judge. Here, the threshold Q is determined according to, for example, the computing power of the decoding device 151 .

If it is determined in step S91 that the priority information is equal to or greater than the threshold Q, the output selection unit 199 supplies the MDCT coefficients of the object to be processed, supplied from the object audio signal decoding unit 198, to the IMDCT unit 201. , the process proceeds to step S92.

In step S<b>92 , the IMDCT unit 201 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 199 to generate an audio signal of the object to be processed, and supplies the audio signal to the rendering unit 162 . After the audio signal is generated, the process proceeds to step S93.

On the other hand, when it is determined in step S91 that the priority information is less than the threshold value Q, the output selection unit 199 sets the MDCT coefficient to 0 and supplies it to the 0 value output unit 200 .

The 0-value output unit 200 generates an audio signal of the object to be processed from the MDCT coefficients of 0 supplied from the output selection unit 199 , and supplies the audio signal to the rendering unit 162 . Therefore, the 0-value output unit 200 does not substantially perform any processing for generating an audio signal such as IMDCT.

Note that the audio signal generated by the 0-value output unit 200 is a silent signal. After the audio signal is generated, the process proceeds to step S93.

If it is determined in step S91 that the priority information is less than the threshold value Q, or if an audio signal is generated in step S92, in step S93 the object audio signal acquisition unit 197 adds 1 to the retained object number. In addition, update the object number of the object being processed.

After the object number is updated, the process returns to step S89 and the above-described processes are repeated. That is, the audio signal of the new object to be processed is generated.

Also, if it is determined in step S89 that the object number of the object to be processed is not less than N, the selective decoding process ends because audio signals have been obtained for all channels and objects, and then the process continues as shown in FIG. to step S53.

As described above, the decoding device 151 compares the priority information and the threshold for each channel or each object, and decodes the encoded audio signal for each channel or object of the frame to be processed. The encoded audio signal is decoded while determining whether or not.

That is, the decoding device 151 decodes a predetermined number of encoded audio signals according to the priority information of each audio signal, and does not decode the remaining audio signals.

As a result, it is possible to selectively decode only audio signals with a high priority according to the reproduction environment, thereby minimizing the deterioration of the sound quality of the sound reproduced by the audio signals and reducing the computational complexity of decoding. be able to.

Moreover, by decoding the encoded audio signal based on the priority information of the audio signal of each object, not only the computational complexity of decoding the audio signal, but also the subsequent processing such as the processing in the rendering unit 162 etc. can also be reduced.

<Modification 1 of the first embodiment>
<About priority information>
In the above description, one piece of priority information is generated for one audio signal of each channel or each object, but a plurality of pieces of priority information may be generated.

In such a case, for example, a plurality of pieces of priority information are generated for each computational capacity according to the computational complexity of decoding, that is, the computational capacity of the decoding side.

Specifically, for example, based on the amount of calculation for decoding audio signals corresponding to two channels in real time, priority information is generated for a device having a calculation capability corresponding to two channels.

For such priority information for a device equivalent to two channels, for example, among all audio signals, the priority is assigned so that the number of audio signals with a lower priority, that is, a value close to 0, is assigned as priority information. Information is generated.

Also, for example, based on the amount of calculation for decoding audio signals corresponding to 24 channels in real time, priority information for devices having calculation capability corresponding to 24 channels is also generated. Priority information for devices with 24 channels is generated so that, among all audio signals, for example, the number of audio signals with a higher priority, that is, a value close to 7, is assigned as priority information. be done.

In this case, for example, in step S11 of FIG. 8, the priority information generation unit 52 generates priority information for devices corresponding to two channels for the audio signal of each channel, and adds 2 to the priority information. An identifier indicating that it is for a device corresponding to a channel is added, and supplied to the packing unit 24 .

Furthermore, in step S11, the priority information generation unit 52 also generates priority information for devices corresponding to 24 channels for the audio signals of each channel, and adds priority information for devices corresponding to 24 channels to the priority information. It is supplied to the packing unit 24 after adding an identifier indicating that it is intended for the purpose.

Similarly, in step S13 of FIG. 8, the priority information generation unit 92 also generates priority information for a device corresponding to 2 channels and priority information for a device corresponding to 24 channels, and adds an identifier. , to the packing section 24 .

As a result, a plurality of pieces of priority information can be obtained according to the computing power of playback devices such as portable audio players, multifunctional mobile phones, tablet computers, television receivers, personal computers, and high-quality audio equipment. become.

For example, a playback device such as a portable audio player has relatively low computational power, so if such a playback device decodes an audio signal encoded based on priority information for two channels worth of devices, Playback of audio signals can be performed in real time.

As described above, when a plurality of pieces of priority information are generated for one audio signal, in the decoding device 151, for example, a higher-level control device uses which priority information among the plurality of pieces of priority information. The priority information acquisition unit 191 or the like is instructed as to whether or not to perform decoding. The indication of which priority information to use is made, for example, by supplying an identifier.

It should be noted that the identifier priority information to be used may be determined in advance for each decoding device 151 .

For example, in the priority information acquisition unit 191, when the priority information of which identifier is to be used is determined in advance, or when the identifier is specified by the upper control device, in step S81 of FIG. 12, the priority information acquisition unit 191 acquires priority information to which a predetermined identifier is added. Then, the acquired priority information is supplied from the priority information acquisition section 191 to the output selection section 194 and the output selection section 199 .

In other words, from among the plurality of priority information stored in the bitstream, one appropriate priority information is selected according to the computing power of the decoding device 151, more specifically, the unpacking/decoding unit 161. be done.

In this case, the priority information may be read from the bitstream using different identifiers for the priority information of each channel and the priority information of each object.

In this way, by selecting and acquiring specific priority information from a plurality of pieces of priority information included in the bitstream, appropriate priority information can be obtained according to the computing power of the decoding device 151. It can be selected and decrypted. As a result, any decoding device 151 can reproduce the audio signal in real time.

<Second embodiment>
<Configuration example of unpacking/decoding section>
In the above description, an example in which the bitstream output from the encoding device 11 includes priority information has been described, but depending on the encoding device, the bitstream may not include priority information. could be.

Therefore, priority information may be generated in the decoding device 151 . For example, it is possible to generate the priority information using information indicating the sound pressure of the audio signal or information indicating the spectrum shape, which can be extracted from the encoded data of the audio signal included in the bitstream.

When generating priority information in the decoding device 151 in this way, the unpacking/decoding unit 161 of the decoding device 151 is configured as shown in FIG. 13, for example. 13, parts corresponding to those in FIG. 10 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

The unpacking/decoding unit 161 shown in FIG. 13 includes a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, an object audio signal acquisition unit 197, an object audio It has a signal decoding section 198 , an output selection section 199 , a 0 value output section 200 , an IMDCT section 201 , a priority information generation section 231 and a priority information generation section 232 .

The configuration of unpacking/decoding section 161 shown in FIG. Unlike the unpacking/decoding section 161 of FIG. 10, other configurations are the same as those of the unpacking/decoding section 161 of FIG.

The channel audio signal acquisition unit 192 acquires the encoded data of each channel from the supplied bitstream, and supplies it to the channel audio signal decoding unit 193 and the priority information generation unit 231 .

The priority information generation unit 231 generates priority information for each channel based on the encoded data for each channel supplied from the channel audio signal acquisition unit 192 and supplies the priority information to the output selection unit 194 .

The object audio signal acquisition unit 197 acquires encoded data of each object from the supplied bitstream, and supplies the encoded data to the object audio signal decoding unit 198 and the priority information generation unit 232 . Also, the object audio signal acquisition unit 197 acquires metadata of each object from the supplied bitstream and supplies the metadata to the rendering unit 162 .

The priority information generation unit 232 generates priority information for each object based on the encoded data of each object supplied from the object audio signal acquisition unit 197 and supplies the priority information to the output selection unit 199 .

<Description of selective decryption processing>
When the unpacking/decoding unit 161 has the configuration shown in FIG. 13, the decoding device 151 performs the selective decoding process shown in FIG. 14 as the process corresponding to step S52 of the decoding process shown in FIG. The selective decoding process by the decoding device 151 will be described below with reference to the flowchart of FIG.

In step S131, the priority information generator 231 generates priority information of the audio signal of each channel.

For example, the channel audio signal acquisition unit 192 acquires the encoded data of each channel from the supplied bitstream and supplies it to the channel audio signal decoding unit 193 and the priority information generation unit 231 .

The priority information generation unit 231 generates priority information for each channel based on the encoded data for each channel supplied from the channel audio signal acquisition unit 192 and supplies the priority information to the output selection unit 194 .

For example, the bitstream contains scale factors for obtaining MDCT coefficients, side information, and quantization spectra as coded data of the audio signal. Here, the scale factor is information indicating the sound pressure of the audio signal, and the quantization spectrum is information indicating the spectral shape of the audio signal.

The priority information generation unit 231 generates priority information of the audio signal of each channel based on the scale factor and quantized spectrum included as encoded data of each channel. In this way, if priority information is generated using a scale factor or a quantized spectrum, priority information can be obtained immediately before decoding encoded data. The amount can also be reduced.

The priority information is also generated based on the sound pressure of the audio signal, which is obtained by calculating the mean square value of the MDCT coefficients, and the spectral shape of the audio signal, which is obtained from the peak envelope of the MDCT coefficients. can be In this case, the priority information generating section 231 decodes the encoded data or acquires the MDCT coefficients from the channel audio signal decoding section 193 as appropriate.

After the priority information of each channel is obtained, the processes of steps S132 to S137 are performed. Since these processes are the same as the processes of steps S82 to S87 in FIG. 12, the description thereof is omitted. . However, in this case, since the encoded data of each channel has already been acquired, only the encoded data is decoded in step S134.

Also, when it is determined in step S133 that the channel number is not less than M, in step S138 the priority information generation unit 232 generates priority information of the audio signal of each object.

For example, the object audio signal acquisition unit 197 acquires encoded data of each object from the supplied bitstream, and supplies it to the object audio signal decoding unit 198 and the priority information generation unit 232 . Also, the object audio signal acquisition unit 197 acquires metadata of each object from the supplied bitstream and supplies the metadata to the rendering unit 162 .

The priority information generation unit 232 generates priority information for each object based on the encoded data of each object supplied from the object audio signal acquisition unit 197 and supplies the priority information to the output selection unit 199 . For example, priority information is generated based on scale factors and quantized spectra, as in each channel.

Also, priority information may be generated based on the sound pressure and spectral shape obtained from the MDCT coefficients. In this case, the priority information generator 232 appropriately decodes the encoded data and acquires the MDCT coefficients from the object audio signal decoder 198 .

After the priority information of each object is obtained, the processing of steps S139 to S144 is performed, and the selective decoding processing ends. These processings are the same as the processing of steps S88 to S93 in FIG. Therefore, its description is omitted. However, in this case, since the encoded data of each object has already been acquired, only the encoded data is decoded in step S141.

After the selective decoding process ends, the process proceeds to step S53 in FIG.

As described above, the decoding device 151 generates priority information of the audio signal of each channel and each object based on the encoded data included in the bitstream. By generating the priority information in the decoding device 151 in this way, appropriate priority information for each audio signal can be obtained with a small amount of calculation, and the amount of calculation for decoding and rendering can be reduced. can. It is also possible to minimize the deterioration of the sound quality of the sound reproduced by the audio signal.

Note that the priority information acquisition unit 191 of the unpacking/decoding unit 161 shown in FIG. 10 tried to acquire the priority information of the audio signal of each channel and each object from the supplied bitstream, The priority information may be generated when the priority information cannot be obtained from the . In such a case, the priority information acquisition unit 191 performs the same processing as the priority information generation unit 231 and the priority information generation unit 232, and obtains the priority information of the audio signal of each channel and each object from the encoded data. Generate.

<Third embodiment>
<Regarding the threshold of priority information>
Furthermore, in the above description, priority information is compared with threshold value P and threshold value Q to select audio signals to be decoded, more specifically, MDCT coefficients for IMDCT are selected for each channel and each object. may be dynamically changed for each frame of the audio signal.

For example, the priority information obtaining unit 191 of the unpacking/decoding unit 161 shown in FIG. 10 can obtain the priority information of each channel and each object from the bitstream without decoding.

Therefore, for example, if the priority information acquisition unit 191 reads the priority information of the audio signals of all channels, it is possible to obtain the distribution of the priority information in the frames to be processed. Also, the decoding device 151 knows in advance its own computational ability, such as how many channels it can process simultaneously, that is, in real time.

Therefore, the priority information acquisition unit 191 determines the priority information threshold value P for the processing target frame based on the priority information distribution in the processing target frame and the computational capacity of the decoding device 151. good too.

For example, the threshold value P is determined so that the maximum number of audio signals is decoded within the range that the decoding device 151 can process in real time.

In addition, the priority information acquisition unit 191 can dynamically determine the threshold Q as in the case of the threshold P. In this case, the priority information acquisition unit 191 obtains the distribution of the priority information based on the priority information of the audio signals of all objects, and based on the obtained distribution and the computing power of the decoding device 151, the processing is performed. A threshold Q for the priority information for the frame of interest is defined.

Such determination of the threshold P and the threshold Q can be performed with a relatively small amount of calculation.

By dynamically changing the threshold of the priority information in this way, it is possible to minimize the deterioration of the sound quality of the sound reproduced by the audio signal while decoding in real time. Especially in such a case, there is no need to prepare a plurality of pieces of priority information, and there is no need to provide an identifier for the priority information, so the code amount of the bitstream can be reduced.

<About object metadata>
Furthermore, in the first to third embodiments described above, it was explained that the leading element of the bitstream stores object metadata and priority information for one frame. .

In this case, the syntax of the portion where the object metadata and priority information are stored in the head element of the bitstream is as shown in FIG. 15, for example.

In the example shown in FIG. 15, the object's spatial position information and priority information for only one frame are stored in the object's metadata.

In this example, "num_objects" indicates the number of objects. Also, "object_priority[o]" indicates the priority information of the O-th object. Here, the O-th object is an object specified by an object number.

"position_azimuth[o]" indicates a horizontal angle representing the three-dimensional spatial position of the O-th object as seen from the user who is the viewer, that is, as seen from a predetermined reference position. Also, "position_elevation[o]" indicates the vertical angle representing the three-dimensional spatial position of the O-th object as seen from the user who is the viewer. Furthermore, "position_radius[o]" indicates the distance from the viewer to the Oth object.

Therefore, the position of the object in the three-dimensional space is specified from these "position_azimuth[o]", "position_elevation[o]", and "position_radius[o]", and this information is the spatial position of the object. regarded as information.

Also, "gain_factor[o]" indicates the gain of the O-th object.

Thus, the metadata shown in FIG. 15 includes "object_priority[o]", "position_azimuth[o]", "position_elevation[o]", "position_radius[o]", and "gain_factor[o]" for one object. o]” are arranged in order as the data of the object. In the metadata, the data of each object are arranged in the order of the object number of the object, for example.

<Fourth Embodiment>
<Complete Reconstruction of Audio Signals and Noise Caused by Discontinuity>
In the above, when the priority information of each frame (hereinafter particularly referred to as a time frame) for each channel or object read from the bitstream in the decoding device 151 is less than a predetermined threshold value, IMDCT etc. An example of reducing the processing amount at the time of decoding by omitting the decoding processing of . Specifically, when the priority information is less than the threshold, the 0-value output unit 195 and the 0-value output unit 200 output a silent audio signal, that is, output 0 data as an audio signal.

However, in such a case, the sound quality deteriorates in terms of audibility. Specifically, sound quality deterioration due to complete reconstruction of the audio signal and sound quality deterioration due to the generation of noise such as glitch noise due to discontinuity of the signal occurs.

(Sound quality deterioration due to perfect reconstruction)
For example, if 0 data is output as an audio signal when the priority information is less than the threshold, sound quality deterioration occurs when switching between the output of 0 data and the output of a normal audio signal that is not 0 data.

As described above, in unpacking/decoding section 161, IMDCT section 196 and IMDCT section 201 perform IMDCT on the MDCT coefficients for each time frame read from the bitstream. More specifically, the unpacking/decoding unit 161 generates an audio signal of the current time frame from the IMDCT result or 0 data for the current time frame and the IMDCT result or 0 data for the previous time frame. be.

Here, generation of an audio signal will be described with reference to FIG. Here, the generation of the audio signal of the object will be described as an example, but the generation of the audio signal of each channel is the same. Also, hereinafter, the audio signal output from the 0-value output unit 200 and the audio signal output from the IMDCT unit 201 are also specifically referred to as an IMDCT signal. Similarly, the audio signal output from 0-value output section 195 and the audio signal output from IMDCT section 196 are also referred to as IMDCT signals.

In FIG. 16, the horizontal direction indicates time, and the rectangles marked with the letters "data[n-1]" to "data[n+2]" are the time frames (n -1) to time frames (n+2). Also, the numerical value in the bitstream of each time frame indicates the value of the priority information of the object of that time frame, and in this example, the value of the priority information of each time frame is "7".

Furthermore, in FIG. 16, rectangles marked with the letters “MDCT_coef[q]” (where q=n-1, n, . . . ) each represent the MDCT coefficients of the time frame (q).

Now, assuming that the threshold Q is 4, the value "7" of the priority information of the time frame (n-1) is greater than or equal to the threshold Q. is done. Similarly, since the priority information value "7" for time frame (n) is also equal to or greater than threshold Q, IMDCT is performed on the MDCT coefficients for time frame (n).

As a result, it is assumed that an IMDCT signal OPS11 of time frame (n-1) and an IMDCT signal OPS12 of time frame (n) are obtained.

In this case, the unpacking/decoding unit 161 adds the first half of the IMDCT signal OPS12 of the time frame (n) and the second half of the IMDCT signal OPS11 of the time frame (n-1) one time frame before, Let the audio signal of time frame (n), that is, the audio signal of period FL(n). In other words, the portion of the period FL(n) of the IMDCT signal OPS11 and the portion of the period FL(n) of the IMDCT signal OPS12 are overlap-added to obtain the pre-encoding time frame (n) of the object to be processed. audio signal is reproduced.

Such processing is necessary for completely reconstructing the IMDCT signal into a pre-MDCT signal.

However, in the above-described unpacking/decoding section 161, for example, as shown in FIG. , the IMDCT signal is no longer perfectly reconstructed to the pre-MDCT signal. In other words, if 0 data is used instead of the original signal during overlap addition, the original audio signal cannot be reproduced because it is not completely reconstructed, and the perceived sound quality of the audio signal deteriorates. .

In FIG. 17, the same characters and the like are written in the parts corresponding to those in FIG. 16, and the description thereof will be omitted.

In the example of FIG. 17, the priority information value of the time frame (n-1) is "7", but the priority information of the other time frames (n) to (n+2) is the lowest " 0”.

Therefore, assuming that the threshold value Q=4, the IMDCT section 201 performs IMDCT on the MDCT coefficients for the time frame (n-1) to obtain the IMDCT signal OPS21 for the time frame (n-1). On the other hand, for time frame (n), IMDCT is not performed on the MDCT coefficients, and 0 data output from 0 value output section 200 is used as IMDCT signal OPS22 for time frame (n).

In this case, the first half of the 0 data, which is the IMDCT signal OPS22 of the time frame (n), and the second half of the IMDCT signal OPS21 of the time frame (n-1) one time frame before that are added together to obtain the final An audio signal of time frame (n). That is, the portions of the period FL(n) of the IMDCT signal OPS22 and the IMDCT signal OPS21 are overlap-added to obtain the final audio signal of the time frame (n) of the object to be processed.

Thus, when the output source of the IMDCT signal switches from IMDCT section 201 to 0-value output section 200 or from 0-value output section 200 to IMDCT section 201, the IMDCT signal from IMDCT section 201 is not completely reconstructed, Deterioration of the sound quality on auditory sense occurs.

(Sound quality deterioration due to noise caused by discontinuity)
Also, when the output source of the IMDCT signal is switched from the IMDCT section 201 to the 0-value output section 200 or from the 0-value output section 200 to the IMDCT section 201, the signal is not completely reconstructed. , the signal may become discontinuous at the connection with the IMDCT signal which is set to 0 data. As a result, glitch noise occurs at the discontinuous connection, and the audible sound quality of the audio signal deteriorates.

Furthermore, in order to improve sound quality in unpacking/decoding section 161, SBR (Spectral Band Replication), etc. may be performed.

Various processes are conceivable as post-processing of the IMDCT unit 201 and the 0-value output unit 200, but the following description will continue with SBR as an example.

In SBR, the high frequency components of the original audio signal before encoding are generated from the audio signal obtained by overlap addition, which is the low frequency component, and the high frequency power value stored in the bitstream. be done.

Specifically, an audio signal for one time frame is divided into several sections called time slots, and the audio signal in each time slot is a plurality of low-frequency sub-band signals (hereinafter referred to as low-frequency sub-band signals). ) are band-divided.

Then, based on the low-frequency sub-band signal of each sub-band and the power value of each high-frequency sub-band, a signal of each high-frequency sub-band (hereinafter also referred to as a high-frequency sub-band signal) is generated. . For example, the target high-frequency subband signal is generated by adjusting the power of the low-frequency sub-band signal of a predetermined sub-band according to the power value of the target high-frequency sub-band or by frequency-shifting it. .

Furthermore, the high frequency sub-band signal and the low frequency sub-band signal are combined to generate an audio signal containing high frequency components, and the audio signals containing high frequency components generated for each time slot are combined to produce a high frequency signal. An audio signal of one time frame containing components.

When such SBR is performed in the subsequent stages of IMDCT section 201 and 0-value output section 200, high-frequency components are generated by SBR for the audio signal composed of the IMDCT signal output from IMDCT section 201. FIG. However, since the IMDCT signal output from the 0-value output unit 200 is 0 data, the high frequency component obtained by SBR is also 0 data for the audio signal composed of the IMDCT signal output from the 0-value output unit 200. end up

Then, when the output source of the IMDCT signal switches from the IMDCT section 201 to the 0-value output section 200 or from the 0-value output section 200 to the IMDCT section 201, the connection portion becomes discontinuous even in the high frequency range. There is In such a case, glitch noise occurs, and the perceived sound quality deteriorates.

Therefore, in this technology, by selecting the output destination of the MDCT coefficients in consideration of the preceding and succeeding time frames, and by performing fade-in and fade-out processing on the audio signal, the above-mentioned deterioration in sound quality in terms of hearing is suppressed and the sound quality is improved. I made it

<Regarding the selection of the output destination of the MDCT coefficients considering the previous and next time frames>
First, the selection of the output destination of the MDCT coefficients in consideration of the preceding and succeeding time frames will be described. Although the audio signal of the object will be described here as an example, the same applies to the audio signal of each channel. Also, the processing described below is performed for each object and for each channel.

For example, in the embodiments described above, the output selection unit 199 selectively switches the output destination of the MDCT coefficients of each object based on the priority information of the current time frame. On the other hand, in the present embodiment, the output selection unit 199 selects three consecutive time frames of the current time frame, the time frame immediately before the current time frame, and the time frame one time after the current time frame. Switch the destination of the MDCT coefficients based on the priority information of one time frame. In other words, whether or not to decode encoded data is selected based on the priority information of three consecutive time frames.

Specifically, the output selection unit 199 supplies the MDCT coefficients of the time frame (n) of the object to the IMDCT unit 201 when the conditional expression shown in the following expression (1) is satisfied for the object to be processed.

In equation (1), object_priority[q] (where q=n−1, n, n+1) indicates priority information for each time frame (q), and thre indicates the threshold Q.

Therefore, in a total of three consecutive time frames, the current time frame and the time frames before and after the current time frame, if there is at least one time frame in which the priority information is equal to or greater than the threshold Q, the MDCT coefficient is supplied to The IMDCT section 201 is selected. In this case, decoding of encoded data, more specifically, IMDCT for MDCT coefficients is performed. On the other hand, when the priority information of those three time frames are all less than the threshold Q, the MDCT coefficient is set to 0 and output to the 0 value output section 200 . In this case, decoding of encoded data, more specifically, IMDCT for MDCT coefficients is not substantially performed.

As a result, the audio signal is completely reconstructed from the IMDCT signal as shown in FIG. 18, and the deterioration of the sound quality perceptually is suppressed. In FIG. 18, portions corresponding to those in FIG. 16 are denoted by the same letters and the like, and description thereof will be omitted.

In the example shown on the upper side of FIG. 18, the values of the priority information for each time frame are the same as in the example shown in FIG. For example, if the threshold Q is 4, the priority information of the time frame (n-1) is equal to or higher than the threshold Q in the example shown on the upper side of the figure, but the priority information of the time frame (n) to the time frame (n+2) , the priority information is less than the threshold Q.

Therefore, IMDCT is performed on the MDCT coefficients of time frame (n-1) and time frame (n) from the conditional expression shown in equation (1) to obtain IMDCT signal OPS31 and IMDCT signal OPS32, respectively. On the other hand, in the time frame (n+1) where the conditional expression is not satisfied, the IMDCT is not performed on the MDCT coefficients, and 0 data is used as the IMDCT signal OPS33.

Therefore, the audio signal of the time frame (n), which was not completely reconstructed in the example of FIG. 17, is completely reconstructed in the example shown in the upper part of FIG. . However, in this example, since the audio signal is not completely reconstructed in the next time frame (n+1), the fade-out process described later is performed in time frame (n) and time frame (n+1), resulting in deterioration of sound quality is suppressed.

In addition, in the example shown on the lower side of the figure, the priority information is less than the threshold Q in time frames (n-1) to (n+1), and the priority in time frame (n+2) The information is equal to or greater than the threshold Q.

Therefore, from the conditional expression shown in equation (1), IMDCT is not performed on the MDCT coefficients in the time frame (n) where the conditional expression is not satisfied, and 0 data is used as the IMDCT signal OPS41. On the other hand, IMDCT is performed on the MDCT coefficients of time frame (n+1) and time frame (n+2) to obtain IMDCT signal OPS42 and IMDCT signal OPS43, respectively.

In this example, since the audio signal can be completely reconstructed in the time frame (n+2) when the priority information switches from a value less than the threshold Q to a value greater than the threshold Q, the perceived sound quality is degraded. can be suppressed. However, even in this case, since the audio signal is not completely reconstructed in the time frame (n+1) immediately before that, the fade-in processing described later is performed in the time frame (n+1) and time frame (n+2). As a result, the deterioration of the sound quality on the sense of hearing is suppressed.

Here, the priority information for one time frame is read ahead, and the output destination of the MDCT coefficient is selected from the priority information for three consecutive time frames. Therefore, fade-out processing is performed in the example time frame (n) and time frame (n+1) shown in the upper part of the figure, and the time frame (n+1) and time frame (n+1) shown in the lower part of the figure are faded out. Fade-in processing is performed at frame (n+2).

However, if priority information for two time frames can be prefetched, fade-out processing is performed in the example time frame (n+1) and time frame (n+2) shown in the upper part of the figure. Alternatively, the fade-in process may be performed at the time frame (n) and the time frame (n+1) shown on the lower side of the figure.

<Regarding fade-in and fade-out processing>
Next, fade-in processing and fade-out processing for audio signals will be described. Although the audio signal of the object will be described here as an example, the same applies to the audio signal of each channel. Also, fade-in processing and fade-out processing are performed for each object and each channel.

In the present technology, as in the example shown in FIG. 18, fade-in is performed in a time frame in which an IMDCT signal obtained by IMDCT and an IMDCT signal that is 0 data are overlap-added and in a time frame before or after that. processing or fade-out processing is performed.

In fade-in processing, gain adjustment is performed on the audio signal so that the amplitude (magnitude) of the audio signal in that time frame increases over time. Conversely, in fade-out processing, gain adjustment is performed on the audio signal so that the amplitude of the audio signal in that time frame decreases over time.

As a result, even when the connecting portion between the IMDCT signal obtained by the IMDCT and the IMDCT signal with 0 data is discontinuous, it is possible to suppress the deterioration of the audible sound quality. Note that hereinafter, the gain value by which the audio signal is multiplied during such gain adjustment is also referred to as a fading signal gain.

Furthermore, in the present technology, fade-in processing or fade-out processing is also performed in SBR on the connecting portion between the IMDCT signal obtained by IMDCT and the IMDCT signal that is 0 data.

That is, in SBR, the power value of each high-frequency subband is used for each time slot, but in this technology, the gain value determined for each time slot for fade-in processing or fade-out processing is used for each high-frequency subband. SBR is performed by multiplying the subband power values.
That is, the gain adjustment of the power value of the high frequency is performed.

In addition, hereinafter, the gain value determined for each time slot, which is multiplied by the high-frequency power value, is also referred to as a fading SBR gain.

Specifically, the fading SBR gain for fade-in processing is determined so that the gain value increases with time, that is, the fading SBR gain of the later time slot has a greater value. It is Conversely, the fading SBR gain for fade-out processing is determined such that the value of the fading SBR gain in the later time slot becomes smaller.

In this way, by performing fade-in processing and fade-out processing even during SBR, it is possible to suppress the deterioration of the audible sound quality even when the high frequencies become discontinuous.

Specifically, the processes shown in FIGS. 19 and 20, for example, are performed as gain adjustments such as fade-in processing and fade-out processing for such audio signals and high-frequency power values. 19 and 20, portions corresponding to those in FIG. 18 are denoted by the same characters, symbols, etc., and description thereof will be omitted.

The example shown in FIG. 19 is an example of the case shown on the upper side in FIG. In this example, the audio signals of time frame (n) and time frame (n+1) are multiplied by the fading signal gain indicated by line GN11.

The value of the fading signal gain indicated by the polygonal line GN11 varies linearly with time from '1' to '0' in the portion of time frame (n), and continues to ' 0”. Therefore, by adjusting the gain of the audio signal using the fading signal gain, the audio signal gradually changes to 0 data, so that the deterioration of the sound quality perceptually can be suppressed.

Also, in this example, the high frequency power value of each time slot of time frame (n) is multiplied by the fading SBR gain indicated by arrow GN12.

The value of the fading SBR gain indicated by the arrow GN12 varies from "1" to "0" so as to become smaller in the later time slots. Therefore, the high-frequency component of the audio signal gradually changes to 0 data by adjusting the high-frequency gain using the fading SBR gain, so it is possible to suppress the deterioration of the audible sound quality.

On the other hand, the example shown in FIG. 20 is an example of the case shown on the lower side in FIG. In this example, the audio signals of time frame (n+1) and time frame (n+2) will be multiplied by the fading signal gain indicated by polygonal line GN21.

The value of the fading signal gain indicated by the polygonal line GN21 is continuously "0" in the portion of the time frame (n+1), and gradually changes from "0" to "0" in the portion of the time frame (n+2). It changes linearly up to "1". Therefore, by adjusting the gain of the audio signal using the fading signal gain, the audio signal gradually changes from 0 data to the original signal.

Also, in this example, the high frequency power value of each time slot of the time frame (n+2) is multiplied by the fading SBR gain indicated by the arrow GN22.

The value of the fading SBR gain indicated by the arrow GN22 varies from "0" to "1" so as to increase in the later time slots. Therefore, by adjusting the high-frequency gain using the fading SBR gain, the high-frequency component of the audio signal gradually changes from 0 data to the original signal.

<Configuration example of unpacking/decoding section>
When the selection of the output destination of the MDCT coefficients described above and the gain adjustment such as fade-in processing and fade-out processing are performed, the unpacking/decoding section 161 is configured as shown in FIG. 21, for example. 21, parts corresponding to those in FIG. 10 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

The unpacking/decoding unit 161 shown in FIG. 21 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and an overlap addition. Section 271, gain adjustment section 272, SBR processing section 273, object audio signal acquisition section 197, object audio signal decoding section 198, output selection section 199, 0 value output section 200, IMDCT section 201, overlap addition section 274, gain adjustment It is composed of a section 275 and an SBR processing section 276 .

The configuration of the unpacking/decoding section 161 shown in FIG. 21 is the same as the configuration of the unpacking/decoding section 161 shown in FIG.

The overlap adder 271 generates an audio signal for each time frame by performing overlap addition on the IMDCT signal (audio signal) supplied from the 0 value output unit 195 or the IMDCT unit 196, and supplies the audio signal to the gain adjuster 272. do.

The gain adjustment unit 272 adjusts the gain of the audio signal supplied from the overlap addition unit 271 based on the priority information supplied from the priority information acquisition unit 191 and supplies the audio signal to the SBR processing unit 273 .

The SBR processing unit 273 acquires the power value of each high frequency sub-band for each time slot from the priority information acquisition unit 191, and based on the priority information supplied from the priority information acquisition unit 191, the SBR processing unit 273 Adjust the gain of the power value. In addition, the SBR processing unit 273 performs SBR on the audio signal supplied from the gain adjustment unit 272 using the gain-adjusted high-frequency power value, and sends the resulting audio signal to the mixing unit 163. supply.

The overlap adder 274 generates an audio signal for each time frame by performing overlap addition on the IMDCT signal (audio signal) supplied from the 0 value output unit 200 or the IMDCT unit 201, and supplies the audio signal to the gain adjuster 275. do.

The gain adjustment unit 275 adjusts the gain of the audio signal supplied from the overlap addition unit 274 based on the priority information supplied from the priority information acquisition unit 191 and supplies the audio signal to the SBR processing unit 276 .

The SBR processing unit 276 acquires the power value of each high frequency sub-band for each time slot from the priority information acquisition unit 191, and based on the priority information supplied from the priority information acquisition unit 191, the SBR processing unit 276 Adjust the gain of the power value. In addition, the SBR processing unit 276 performs SBR on the audio signal supplied from the gain adjustment unit 275 using the gain-adjusted high-frequency power value, and outputs the resulting audio signal to the rendering unit 162. supply.

<Description of selective decryption processing>
Next, the operation of decoding device 151 when unpacking/decoding section 161 has the configuration shown in FIG. 21 will be described. In this case, the decoding device 151 performs the decoding process described with reference to FIG. However, as the selective decoding process in step S52, the process shown in FIG. 22 is performed.

The selective decoding process corresponding to the process of step S52 in FIG. 11 will be described below with reference to the flowchart in FIG.

In step S181, the priority information acquisition unit 191 acquires the high frequency power value of the audio signal of each channel from the supplied bitstream and supplies it to the SBR processing unit 273, and also acquires the power value of each object from the bitstream. A high frequency power value of the audio signal is acquired and supplied to the SBR processing unit 276 .

After the high-frequency power value is obtained, the processing of steps S182 to S187 is performed to generate the audio signal (IMDCT signal) of the channel to be processed. Since it is the same as the processing in step S86, the explanation thereof is omitted.

However, in step S186, if a conditional expression similar to the above-described expression (1) is satisfied, that is, the priority information of the current time frame of the channel to be processed, and the priority information of the time frames immediately before and after the current time frame. If even one piece of priority information is greater than or equal to the threshold P, it is determined that the priority information is greater than or equal to the threshold P. Also, the IMDCT signal generated by the 0-value output section 195 or the IMDCT section 196 is output to the overlap addition section 271 .

If it is not determined in step S186 that the priority information is equal to or greater than the threshold value P, or if the IMDCT signal is generated in step S187, the process of step S188 is performed.

In step S188, the overlap adder 271 performs overlap addition of the IMDCT signals supplied from the 0-value output unit 195 or the IMDCT unit 196, and sends the resulting audio signal of the current time frame to the gain adjuster 272. supply.

Specifically, for example, as described with reference to FIG. 18, the first half of the IMDCT signal of the current time frame and the second half of the IMDCT signal of the previous time frame are added together to form the audio signal of the current time frame. be done.

In step S189, the gain adjustment unit 272 adjusts the gain of the audio signal supplied from the overlap addition unit 271 based on the priority information of the processing target channel supplied from the priority information acquisition unit 191, and performs SBR processing. 273.

Specifically, the gain adjustment unit 272 determines that the priority information of the time frame immediately before the current time frame is equal to or greater than the threshold value P, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame If the information is less than the threshold P, the gain of the audio signal is adjusted with the fading signal gain indicated by the polygonal line GN11 in FIG. In this case, the time frame (n) in FIG. 19 corresponds to the current time frame, and in the time frame immediately after the current time frame, gain adjustment is performed with the fading signal gain=0, as indicated by the polygonal line GN11. .

If the priority information of the current time frame is equal to or greater than the threshold value P and the priority information of the two time frames immediately before the current time frame are both less than the threshold value P, the gain adjustment unit 272 shifts to the polygonal line GN21 in FIG. Adjust the gain of the audio signal with the indicated fading signal gain. In this case, the time frame (n+2) in FIG. 20 corresponds to the current time frame, and in the time frame immediately preceding the current time frame, gain adjustment with fading signal gain=0 is performed as indicated by the polygonal line GN21. done.

Note that the gain adjustment unit 272 performs gain adjustment only in these two cases, and does not perform gain adjustment in other cases, and supplies the audio signal to the SBR processing unit 273 as it is.

In step S190, the SBR processing unit 273 converts the audio signal supplied from the gain adjustment unit 272 into SBR against it.

Specifically, the SBR processing unit 273 determines that the priority information of the time frame immediately before the current time frame is equal to or greater than the threshold value P, and that the priority information of the current time frame and the priority of the time frame immediately after the current time frame If the degree information is less than the threshold value P, the power value of the high frequency range is gain-adjusted with the fading SBR gain indicated by the arrow GN12 in FIG. That is, the high frequency power value is multiplied by the fading SBR gain.

Then, the SBR processing unit 273 performs SBR using the gain-adjusted high-frequency power value, and supplies the resulting audio signal to the mixing unit 163 . In this case, time frame (n) in FIG. 19 corresponds to the current time frame.

If the priority information of the current time frame is equal to or greater than the threshold value P and the priority information of two time frames immediately preceding the current time frame are both less than the threshold value P, the SBR processing unit 273 moves to arrow GN22 in FIG. Gain adjust the high frequency power value with the indicated fading SBR gain. Then, the SBR processing unit 273 performs SBR using the gain-adjusted high-frequency power value, and supplies the resulting audio signal to the mixing unit 163 . In this case, time frame (n+2) in FIG. 20 corresponds to the current time frame.

Note that the SBR processing unit 273 performs gain adjustment of the high-frequency power value only in these two cases, and in other cases, does not adjust the gain, and uses the acquired high-frequency power value as it is. SBR is performed using the SBR, and the audio signal obtained as a result is supplied to the mixing unit 163 .

After the SBR is performed and the audio signal of the current time frame is obtained, the processing of steps S191 to S196 is performed. The explanation is omitted.

However, in step S195, it is determined that the priority information is equal to or greater than the threshold value Q when the conditional expression (1) described above is satisfied. Also, the IMDCT signal (audio signal) generated by the 0-value output section 200 or the IMDCT section 201 is output to the overlap addition section 274 .

When the IMDCT signal of the current time frame is obtained in this way, the processes of steps S197 to S199 are performed to generate the audio signal of the current time frame, but these processes are the same as the processes of steps S188 to S190. Since it is the same, its explanation is omitted.

In step S200, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S193. Then, if it is determined in step S193 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S53 in FIG.

As described above, the unpacking/decoding unit 161 selects the output destination of the MDCT coefficients according to the priority information of the current time frame and the time frames before and after it. As a result, the audio signal is completely reconfigured at the transition between time frames in which the priority information is above the threshold and time frames in which the priority information is below the threshold, thereby suppressing deterioration in perceived sound quality. can do.

Also, the unpacking/decoding unit 161 gain-adjusts the audio signal after overlap addition and the power value of the high frequency band based on the priority information of the three consecutive time frames. That is, fade-in processing and fade-out processing are performed as appropriate. As a result, it is possible to suppress the occurrence of glitch noise and suppress the deterioration of sound quality in terms of audibility.

<Fifth embodiment>
<Regarding fade-in and fade-out processing>
In the fourth embodiment, it has been explained that the gain adjustment is performed on the audio signal after overlap addition, and further the gain adjustment is performed on the high frequency power value during SBR. In this case, gain adjustment, that is, fade-in processing and fade-out processing are performed separately for the low-frequency component and the high-frequency component of the final audio signal.

Therefore, in order to realize fade-in processing and fade-out processing with less processing, gain adjustment is not performed immediately after overlap addition and during SBR, but gain adjustment is performed on the audio signal obtained by SBR. You may do so.

In such a case, gain adjustment is performed as shown in FIGS. 23 and 24, for example. In FIGS. 23 and 24, portions corresponding to those in FIGS. 19 and 20 are denoted by the same letters and the like, and description thereof will be omitted.

The example shown in FIG. 23 is an example in which the change in priority information is the same as in the case shown in FIG. In this example, assuming that the threshold Q=4, the priority information in the time frame (n-1) is equal to or greater than the threshold Q, but in the time frames (n) to (n+2), the priority information is less than the threshold Q.

In such a case, the audio signal obtained by SBR at time frame (n) and time frame (n+1) is multiplied by the fading signal gain indicated by line GN31 to adjust the gain. Become.

The fading signal gain indicated by this polygonal line GN31 is the same as the fading signal gain indicated by the polygonal line GN11 in FIG. However, in the case of the example of FIG. 23, the audio signal to be gain-adjusted includes both low-frequency and high-frequency components. Adjustments can be made with one fading signal gain.

By adjusting the gain of the audio signal by such fading signal gain, the audio signal gradually changes at the portion where the IMDCT signal obtained by the IMDCT and the IMDCT signal with 0 data are overlap-added and the portion immediately before that. gradually changes to 0 data. As a result, it is possible to suppress the deterioration of the sound quality in terms of audibility.

On the other hand, the example shown in FIG. 24 is an example in which the change in priority information is the same as in the case shown in FIG. In this example, assuming threshold Q=4, the priority information for time frame (n) and time frame (n+1) is less than threshold Q, but the priority information for time frame (n+2) is It is equal to or greater than the threshold Q.

In such a case, the audio signal obtained by SBR in time frame (n+1) and time frame (n+2) is multiplied by the fading signal gain indicated by polygonal line GN41 to adjust the gain. It will be.

The fading signal gain indicated by this polygonal line GN41 is the same as the fading signal gain indicated by the polygonal line GN21 in FIG. However, in the case of the example of FIG. 24, the audio signal to be gain-adjusted includes both low-frequency and high-frequency components. Adjustments can be made with one fading signal gain.

By adjusting the gain of the audio signal using such fading signal gain, the audio signal becomes 0 at the portion where the IMDCT signal obtained by the IMDCT and the IMDCT signal with 0 data are overlap-added and the portion immediately after that. The data gradually changes to the original signal. As a result, it is possible to suppress the deterioration of the sound quality in terms of audibility.

<Configuration example of unpacking/decoding section>
When gain adjustment is performed by the fade-in processing and fade-out processing described with reference to FIGS. 23 and 24, the unpacking/decoding section 161 is configured as shown in FIG. 25, for example. In FIG. 25, parts corresponding to those in FIG. 21 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

The unpacking/decoding unit 161 shown in FIG. 25 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and an overlap addition. Unit 271, SBR processing unit 273, gain adjustment unit 272, object audio signal acquisition unit 197, object audio signal decoding unit 198, output selection unit 199, 0 value output unit 200, IMDCT unit 201, overlap addition unit 274, SBR processing It is composed of a section 276 and a gain adjustment section 275 .

The configuration of unpacking/decoding section 161 shown in FIG. 25 is similar to that shown in FIG. 21 in that gain adjustment section 272 and gain adjustment section 275 are arranged after SBR processing section 273 and SBR processing section 276, respectively. It differs from the configuration of the unpacking/decoding section 161 .

In the unpacking/decoding unit 161 shown in FIG. 25, the SBR processing unit 273 performs The audio signal obtained as a result of the SBR is supplied to the gain adjustment section 272 . In this case, the SBR processing section 273 does not adjust the gain of the high frequency power value.

The gain adjustment unit 272 adjusts the gain of the audio signal supplied from the SBR processing unit 273 based on the priority information supplied from the priority information acquisition unit 191 and supplies the audio signal to the mixing unit 163 .

The SBR processing unit 276 performs SBR on the audio signal supplied from the overlap addition unit 274 based on the high frequency power value supplied from the priority information acquisition unit 191, and the resulting audio signal is supplied to the gain adjustment unit 275 . In this case, the SBR processing section 276 does not adjust the gain of the high frequency power value.

The gain adjustment unit 275 adjusts the gain of the audio signal supplied from the SBR processing unit 276 based on the priority information supplied from the priority information acquisition unit 191 and supplies the result to the rendering unit 162 .

<Description of selective decryption processing>
Next, the operation of decoding device 151 when unpacking/decoding section 161 has the configuration shown in FIG. 25 will be described. In this case, the decoding device 151 performs the decoding process described with reference to FIG. However, as the selective decoding process in step S52, the process shown in FIG. 26 is performed.

The selective decoding process corresponding to the process of step S52 in FIG. 11 will be described below with reference to the flowchart in FIG. Note that the processing from step S231 to step S238 is the same as the processing from step S181 to step S188 in FIG. 22, so description thereof will be omitted. However, priority information is not supplied to the SBR processing unit 273 and the SBR processing unit 276 in step S232.

In step S239, the SBR processing unit 273 performs SBR on the audio signal supplied from the overlap addition unit 271 based on the high frequency power value supplied from the priority information acquisition unit 191, and obtains The obtained audio signal is supplied to the gain adjustment section 272 .

In step S<b>240 , the gain adjustment unit 272 gain-adjusts the audio signal supplied from the SBR processing unit 273 based on the priority information of the processing target channel supplied from the priority information acquisition unit 191 . supply to

Specifically, the gain adjustment unit 272 determines that the priority information of the time frame immediately before the current time frame is equal to or greater than the threshold value P, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame If the information is less than the threshold P, adjust the gain of the audio signal with the fading signal gain indicated by the polygonal line GN31 in FIG. In this case, the time frame (n) in FIG. 23 corresponds to the current time frame, and in the time frame immediately after the current time frame, gain adjustment is performed with the fading signal gain=0, as indicated by the polygonal line GN31. .

If the priority information of the current time frame is equal to or greater than the threshold value P and the priority information of the two time frames immediately preceding the current time frame are both less than the threshold value P, the gain adjustment unit 272 shifts to the polygonal line GN41 in FIG. Adjust the gain of the audio signal with the indicated fading signal gain. In this case, the time frame (n+2) in FIG. 24 corresponds to the current time frame, and in the time frame immediately preceding the current time frame, gain adjustment is performed with the fading signal gain=0, as indicated by the polygonal line GN41. done.

Note that the gain adjustment section 272 performs gain adjustment only in these two cases, and does not perform gain adjustment in other cases, and supplies the audio signal to the mixing section 163 as it is.

After the gain adjustment of the audio signal is performed, the processes of steps S241 to S247 are performed, but since these processes are the same as the processes of steps S191 to S197 in FIG. 22, the description thereof will be omitted.

When the audio signal of the current time frame of the object to be processed is obtained in this way, the processes of steps S248 and S249 are performed to generate the final audio signal of the current time frame. Since it is the same as the processing in steps S239 and S240, the description thereof will be omitted.

In step S250, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S243. Then, if it is determined in step S243 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S53 in FIG.

As described above, the unpacking/decoding unit 161 adjusts the gain of the audio signal obtained by SBR based on the priority information of the three consecutive time frames. As a result, it is possible to more easily suppress the occurrence of glitch noise and suppress the deterioration of the sound quality on the sense of hearing.

In this embodiment, an example of selecting the output destination of the MDCT coefficient using priority information for three time frames and adjusting the gain by the fading signal gain has been described. Alternatively, only adjustments may be made.

In such a case, the output selection section 194 and the output selection section 199 select the output destination of the MDCT coefficients by the same processing as in the first embodiment. Then, when the priority information of the current time frame is less than the threshold, the gain adjustment section 272 and the gain adjustment section 275 linearly increase or decrease the fading signal gain of the current time frame to perform fade-in processing and fade-out processing. process. Here, whether to perform fade-in processing or fade-out processing may be determined based on the priority information of the current time frame and the priority information of time frames before and after it.

<Sixth Embodiment>
<Regarding fade-in and fade-out processing>
By the way, in the rendering unit 162, for example, VBAP is performed to generate an audio signal of each channel for reproducing the sound of each object from the audio signal of each object.

Specifically, in VBAP, an audio signal gain value (hereinafter also referred to as VBAP gain) for each object is calculated for each channel, that is, for each speaker that outputs audio, for each time frame. Then, the sum of the audio signals of each object multiplied by the VBAP gain for the same channel (speaker) is taken as the audio signal of that channel. In other words, for each object, the audio signal of the object is assigned to each of their channels with the VBAP gain calculated for each channel.

Therefore, for the object audio signal, instead of adjusting the gain of the object audio signal and the power value of the high frequency, by appropriately adjusting the VBAP gain, the occurrence of glitch noise is suppressed and the sound quality is improved. deterioration may be suppressed.

In such a case, for example, linear interpolation or the like is performed on the VBAP gain of each time frame, the VBAP gain for each sample of the audio signal in each time frame is calculated, and the obtained VBAP gain is used to calculate the audio signal of each channel. is generated.

For example, the VBAP gain value of the leading sample of the time frame to be processed is the VBAP gain value of the last sample of the time frame immediately preceding the time frame to be processed. Also, the value of the VBAP gain of the sample at the end of the time frame to be processed is the value of the VBAP gain calculated by the normal VBAP for the time frame to be processed.

Then, in the time frame to be processed, the VBAP gain value of each sample between the leading sample and the trailing sample is determined such that the VBAP gain varies linearly from the leading sample to the trailing sample.

However, if the priority information of the time frame to be processed is less than the threshold, the VBAP is not calculated, and the value of the VBAP gain of the sample at the end of the time frame to be processed is set to zero. Then, the VBAP gain of each sample is determined such that the VBAP gain varies linearly from the first sample to the last sample of the time frame to be processed.

By adjusting the gain of the audio signal of each object using the VBAP gain in this way, it is possible to adjust the gain of the low-frequency component and the high-frequency component at once, suppressing the occurrence of glitch noise with a smaller amount of processing. It is possible to suppress the deterioration of the sound quality on the sense of hearing.

When the VBAP gain is determined for each sample in this way, the VBAP gain for each sample in each time frame is as shown in FIGS. 27 and 28, for example.

In FIGS. 27 and 28, portions corresponding to those in FIGS. 19 and 20 are denoted by the same letters, etc., and description thereof will be omitted. 27 and 28, "VBAP_gain[q][s]" (where q = n-1, n, n+1, n+2) is a speaker Fig. 3 shows the VBAP gain for the time frame (q) of the object being processed with index s;

The example shown in FIG. 27 is an example in which the change in priority information is the same as the case shown in FIG. In this example, assuming that the threshold Q=4, the priority information in the time frame (n-1) is equal to or greater than the threshold Q, but in the time frames (n) to (n+2), the priority information is less than the threshold Q.

In such a case, the VBAP gains from time frame (n−1) to time frame (n+1) are, for example, the gains indicated by polygonal line GN51.

In this example, the priority information of the time frame (n-1) is equal to or higher than the threshold Q, so the VBAP gain of each sample is determined based on the VBAP gain calculated by normal VBAP.

That is, the VBAP gain value of the sample at the beginning of the time frame (n-1) is the same as the VBAP gain value of the sample at the end of the time frame (n-2). Also, the value of the VBAP gain for the last sample in time frame (n-1) corresponds to the speaker s calculated by the normal VBAP for time frame (n-1) for the object being processed. It is the value of the channel's VBAP gain. The VBAP gain value of each sample in the time frame (n-1) is determined to change linearly from the first sample to the last sample.

Also, since the priority information of time frame (n) is less than the threshold Q, the VBAP gain value of the last sample of time frame (n) is set to zero.

That is, the VBAP gain value for the leading sample of time frame (n) is assumed to be the same as the VBAP gain value for the trailing sample of time frame (n-1), and the VBAP gain value for the trailing sample of time frame (n) is The gain value is set to 0. Then, the value of the VBAP gain of each sample in time frame (n) is determined so as to change linearly from the first sample to the last sample.

Furthermore, since the priority information of time frame (n+1) is less than the threshold Q, the VBAP gain value of the last sample of time frame (n+1) is set to 0, resulting in time frame (n+1 ) becomes 0 for the VBAP gain of all samples.

In this way, by setting the VBAP gain value of the sample at the end of the time frame whose priority information is less than the threshold Q to 0, a fade-out process equivalent to the example in FIG. 23 can be performed.

On the other hand, the example shown in FIG. 28 is an example in which the change in priority information is the same as in the case shown in FIG. In this example, assuming that the threshold Q=4, the priority information is less than the threshold Q in the time frames (n-1) to (n+1), but the priority in the time frame (n+2) is The information is equal to or greater than the threshold Q.

In such a case, the VBAP gains for time frame (n-1) to time frame (n+2) are, for example, the gains indicated by polygonal line GN61.

In this example, both the priority information for time frame (n) and the priority information for time frame (n+1) are below threshold Q, so the VBAP gain for all samples in time frame (n+1) is zero. Become.

Also, since the priority information of the time frame (n+2) is equal to or higher than the threshold Q, for the object to be processed, based on the VBAP gain of the channel corresponding to the speaker s calculated by normal VBAP, A VBAP gain for each sample is determined.

That is, the VBAP gain value of the sample at the beginning of the time frame (n+2) is set to 0, which is the VBAP gain value of the sample at the end of the time frame (n+1), and the VBAP gain value of the sample at the end of the time frame (n+2). The VBAP gain value of the last sample is the VBAP gain value calculated by normal VBAP for time frame (n+2). The value of the VBAP gain of each sample in the time frame (n+2) is determined to change linearly from the first sample to the last sample.

In this way, by setting the value of the VBAP gain of the sample at the end of the time frame whose priority information is less than the threshold Q to 0, fade-in processing equivalent to the example of FIG. 24 can be performed.

<Configuration example of unpacking/decoding section>
When gain adjustment is performed by the fade-in processing and fade-out processing described with reference to FIGS. 27 and 28, the unpacking/decoding section 161 is configured as shown in FIG. 29, for example. In FIG. 29, parts corresponding to those in FIG. 25 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

The unpacking/decoding unit 161 shown in FIG. 29 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and an overlap addition. Unit 271, SBR processing unit 273, gain adjustment unit 272, object audio signal acquisition unit 197, object audio signal decoding unit 198, output selection unit 199, 0 value output unit 200, IMDCT unit 201, overlap addition unit 274, and SBR It is composed of a processing unit 276 .

The configuration of unpacking/decoding section 161 shown in FIG. 29 differs from the configuration of unpacking/decoding section 161 shown in FIG. 25 in that gain adjustment section 275 is not provided, and otherwise has the same configuration. ing.

In the unpacking/decoding unit 161 shown in FIG. 29, the SBR processing unit 276 performs SBR is performed using the SBR, and the audio signal obtained as a result is supplied to the rendering unit 162 .

Also, the priority information acquisition unit 191 acquires metadata and priority information of each object from the supplied bitstream and supplies them to the rendering unit 162 . The priority information of each object is also supplied to the output selection section 199 .

<Description of Decryption Processing>
Next, the operation of decoding device 151 when unpacking/decoding section 161 has the configuration shown in FIG. 29 will be described.

In this case, the decoding device 151 performs the decoding process shown in FIG. The decoding process performed by the decoding device 151 will be described below with reference to the flowchart of FIG. However, in step S281, the same processing as the processing in step S51 of FIG. 11 is performed, so the description thereof will be omitted.

In step S282, the unpacking/decoding unit 161 performs selective decoding processing.

Here, the selective decoding process corresponding to the process of step S282 in FIG. 30 will be described with reference to the flowchart in FIG.

Note that the processing from step S311 to step S328 is the same as the processing from step S231 to step S248 in FIG. 26, so description thereof will be omitted. However, in step S<b>312 , the priority information acquisition unit 191 also supplies the priority information acquired from the bitstream to the rendering unit 162 .

In step S329, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S323. Then, if it is determined in step S323 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S283 in FIG.

Therefore, in the selective decoding process shown in FIG. 31, the audio signal of each channel is subjected to gain adjustment based on the fading signal gain as in the fifth embodiment, and the gain adjustment is not performed for each object. The audio signal obtained by SBR is output to the rendering unit 162 as it is.

Returning to the description of the decoding process in FIG. 30, in step S283, the rendering unit 162 converts the audio signal of each object supplied from the SBR processing unit 276 into the metadata of each object supplied from the priority information acquisition unit 191. and the priority information of the current time frame of each object, the audio signal of each object is rendered.

For example, as described with reference to FIGS. 27 and 28, the rendering unit 162, for each object, for each channel, the priority information of the current time frame and the VBAP of the sample at the end of the time frame immediately before the current time frame. Based on the gain, calculate the VBAP gain for each sample in the current time frame. At this time, the rendering unit 162 appropriately calculates the VBAP gain by VBAP based on the position information.

Then, the rendering unit 162 generates an audio signal of each channel based on the VBAP gain for each sample of each channel calculated for each object and the audio signal of each object, and supplies the audio signal to the mixing unit 163 .

Although the example of calculating the VBAP gain of each sample so that the VBAP gain of each sample in the time frame changes linearly has been described here, the VBAP gain may change nonlinearly. In addition, an example in which the audio signal of each channel is generated by VBAP has been explained, but even when the audio signal of each channel is generated by other methods, the gain of the audio signal of each object is calculated by the same processing as in VBAP. can be adjusted.

After the audio signal of each channel is generated, the process of step S284 is performed and the decoding process ends. However, since the process of step S284 is the same as the process of step S54 in FIG. 11, the description thereof is omitted. do.

In this way, the decoding device 151 calculates the VBAP gain for each sample based on the priority information for each object, and adjusts the gain of the audio signal of the object using the VBAP gain when generating the audio signal of each channel. As a result, it is possible to suppress the occurrence of glitch noise with a smaller amount of processing, and to suppress the deterioration of the sound quality on the sense of hearing.

It should be noted that in the fourth to sixth embodiments, the priority information of the time frames immediately before and after the current time frame is used to select the output destination of the MDCT coefficients, the fading signal gain, etc. He explained that gain adjustment is performed by However, not limited to this, the priority information of the current time frame, the priority information of the time frame preceding the current time frame by a predetermined time frame, and the priority information of the time frame following the current time frame by a predetermined time frame. and may be used.

By the way, the series of processes described above can be executed by hardware or by software. When executing a series of processes by software, a program that constitutes the software is installed in the computer. Here, the computer includes, for example, a computer built into dedicated hardware and a general-purpose computer capable of executing various functions by installing various programs.

FIG. 32 is a block diagram showing a hardware configuration example of a computer that executes the series of processes described above by a program.

In the computer, a CPU (Central Processing Unit) 501 , a ROM (Read Only Memory) 502 and a RAM (Random Access Memory) 503 are interconnected by a bus 504 .

An input/output interface 505 is also connected to the bus 504 . An input unit 506 , an output unit 507 , a recording unit 508 , a communication unit 509 and a drive 510 are connected to the input/output interface 505 .

An input unit 506 includes a keyboard, mouse, microphone, imaging device, and the like. The output unit 507 includes a display, a speaker, and the like. A recording unit 508 is composed of a hard disk, a nonvolatile memory, or the like. A communication unit 509 includes a network interface and the like. A drive 510 drives a removable medium 511 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

In the computer configured as described above, for example, the CPU 501 loads the program recorded in the recording unit 508 into the RAM 503 via the input/output interface 505 and the bus 504 and executes the above-described series of programs. is processed.

A program executed by the computer (CPU 501) can be provided by being recorded on a removable medium 511 such as a package medium, for example. Also, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

In the computer, the program can be installed in the recording section 508 via the input/output interface 505 by loading the removable medium 511 into the drive 510 . Also, the program can be received by the communication unit 509 and installed in the recording unit 508 via a wired or wireless transmission medium. In addition, the program can be installed in the ROM 502 or the recording unit 508 in advance.

The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be executed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

Further, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

For example, the present technology can take a configuration of cloud computing in which one function is shared by a plurality of devices via a network and processed jointly.

Further, each step described in the flowchart above can be executed by one device, or can be shared by a plurality of devices and executed.

Furthermore, when one step includes a plurality of processes, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.

Moreover, the effects described in this specification are only examples and are not limited, and other effects may be provided.

Furthermore, the present technology can also be configured as follows.

(1)
an acquisition unit for acquiring encoded audio signals of multiple channels or multiple objects and priority information of each said audio signal at a predetermined time;
and an audio signal decoding unit that decodes the encoded audio signals of a predetermined number of channels or objects according to the priority information, based on the priority information.
(2)
The decoding device according to (1), wherein the audio signal decoding unit decodes the encoded audio signal having a priority indicated by the priority information equal to or higher than a predetermined level.
(3)
The decoding device according to (2), wherein the obtaining unit changes the predetermined degree based on the priority information of the audio signals of the plurality of channels or the plurality of objects at the predetermined time.
(4)
The acquisition unit acquires a plurality of pieces of priority information for each audio signal,
any one of (1) to (3), wherein the audio signal decoding unit decodes the encoded audio signal based on one of the priority information selected from the plurality of priority information; The decoding device according to item 1.
(5)
(4) The decoding device according to (4), wherein the plurality of pieces of priority information are generated for each computational capability according to the computational capability of the decoding side of the encoded audio signal.
(6)
The decoding device according to any one of (1) to (5), further comprising a priority information generation unit that generates the priority information based on the encoded audio signal.
(7)
The decoding device according to (6), wherein the priority information generating section generates the priority information based on sound pressure or spectrum shape of an audio signal obtained from the encoded audio signal.
(8)
The audio signal decoding unit decodes, for each channel or each object, the predetermined time based on the priority information for the predetermined time and the priority information for the time before or after the predetermined time. The decoding device according to (1), which selects whether to decode the encoded audio signal.
(9)
When the decoding is performed, the signal obtained by the decoding is used as an output signal, and when the decoding is not performed, 0 data is used as the output signal, and the output of the predetermined time is performed for each channel or each object. an addition unit that adds the signal and the output signal before or after the predetermined time to generate the audio signal of the predetermined time;
Adjusting the gain of the audio signal for the predetermined time based on the priority information for the predetermined time and the priority information for the time before or after the predetermined time for each channel or for each object. The decoding device according to (1), further comprising:
(10)
gain-adjusting a high-frequency power value for each channel or for each object based on the priority information for the predetermined time and the priority information for the time before or after the predetermined time; (9) The decoding device according to (9), further comprising a high-frequency generating unit that generates high-frequency components of the audio signal at the predetermined time based on the gain-adjusted power value and the audio signal at the predetermined time. .
(11)
further comprising a high frequency generator that generates an audio signal of the predetermined time containing high frequency components based on the power value of the high frequency and the audio signal of the predetermined time for each channel or object;
The decoding device according to (9), wherein the gain adjustment section adjusts the gain of the audio signal for the predetermined time including high frequency components.
(12)
(1) further comprising a rendering unit that allocates an audio signal of an object to each of the plurality of channels with a predetermined gain value based on the priority information for the predetermined time to generate the audio signal of each of the plurality of channels. The decoding device according to .
(13)
Obtaining encoded audio signals of multiple channels or multiple objects and priority information of each said audio signal at a given time;
decoding, based on the priority information, the encoded audio signal of a predetermined number of channels or objects according to the priority information.
(14)
Obtaining encoded audio signals of multiple channels or multiple objects and priority information of each said audio signal at a given time;
A program that causes a computer to perform a process comprising, based on the priority information, decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information.
(15)
a priority information generator for generating priority information at a given time of audio signals of multiple channels or multiple objects;
and a packing unit that stores the priority information in a bitstream.
(16)
The encoding device according to (15), wherein the priority information generation unit generates a plurality of pieces of the priority information for each audio signal.
(17)
The encoding device according to (16), wherein the priority information generation unit generates the priority information for each computational capability according to the computational capability of a decoding side of the encoded audio signal.
(18)
The encoding device according to any one of (15) to (17), wherein the priority information generating section generates the priority information based on sound pressure or spectrum shape of the audio signal.
(19)
further comprising an encoding unit that encodes audio signals of the plurality of channels or the plurality of objects;
The encoding device according to any one of (15) to (18), wherein the packing unit stores the priority information and the encoded audio signal in the bitstream.
(20)
generating priority information at a given time for an audio signal of multiple channels or multiple objects;
An encoding method, comprising: storing the priority information in a bitstream.
(21)
generating priority information at a given time for an audio signal of multiple channels or multiple objects;
A program that causes a computer to execute a process including a step of storing the priority information in a bitstream.

11 encoding device, 21 channel audio encoding unit, 22 object audio encoding unit, 23 metadata input unit, 24 packing unit, 51 encoding unit, 52 priority information generation unit, 61 MDCT unit, 91 encoding unit, 92 priority information generation unit, 101 MDCT unit, 151 decoding device, 161 unpacking/decoding unit, 162 rendering unit, 163 mixing unit, 191 priority information acquisition unit, 193 channel audio signal decoding unit, 194 output selection unit, 196 IMDCT unit, 198 object audio signal decoding unit, 199 output selection unit, 201 IMDCT unit, 231 priority information generation unit, 232 priority information generation unit, 271 overlap addition unit, 272 gain adjustment unit, 273 SBR processing unit, 274 overlap processor, 275 gain adjuster, 276 SBR processor

TECHNICAL FIELD The present technology relates to a decoding device, method, and program, and more particularly to a decoding device, method, and program capable of reducing the amount of calculation for decoding an audio signal.

A decoding device according to a first aspect of the present technology includes an acquisition unit that acquires an encoded audio signal of a channel or an object and priority information of the audio signal; an audio signal decoding unit that decodes the coded audio signals of a predetermined number of channels or objects according to and performs SBR processing based on the high-frequency power value and the audio signal for each channel or each object an SBR processing unit that generates an audio signal including high-frequency components; and an audio signal of the object generated by the SBR processing is allocated to each of a plurality of channels with a predetermined gain value to generate an audio signal of each of the plurality of channels. and a rendering unit that generates

A decoding method or program according to the first aspect of the present technology acquires an encoded audio signal of a channel or an object and priority information of the audio signal , and converts the priority information to the priority information based on the priority information. Decode the encoded audio signal of a predetermined number of channels or objects according to each channel or object, perform SBR processing based on the power value of the high frequency band and the audio signal, and include the high frequency component. and assigning the audio signal of the object generated by the SBR processing to each of the plurality of channels with a predetermined gain value to generate the audio signal of each of the plurality of channels.

In a first aspect of the present technology, an encoded audio signal of a channel or an object and priority information of the audio signal are obtained, and based on the priority information, a predetermined The encoded audio signal of a number of channels or objects is decoded, SBR processing is performed based on the high frequency power value and the audio signal for each channel or object, and an audio signal containing high frequency components is generated, the audio signal of the object generated by the SBR processing is assigned to each of the plurality of channels with a predetermined gain value, and the audio signal of each of the plurality of channels is generated.

Claims (21)

an acquisition unit for acquiring encoded audio signals of multiple channels or multiple objects and priority information of each said audio signal at a predetermined time;
and an audio signal decoding unit that decodes the encoded audio signals of a predetermined number of channels or objects according to the priority information, based on the priority information. The decoding device according to claim 1, wherein the audio signal decoding section decodes the encoded audio signal whose priority level indicated by the priority level information is equal to or higher than a predetermined level. The decoding device according to claim 2, wherein the obtaining unit changes the predetermined degree based on the priority information of the audio signals of the plurality of channels or the plurality of objects at the predetermined time. The acquisition unit acquires a plurality of pieces of priority information for each audio signal,
The decoding device according to claim 1, wherein the audio signal decoding section decodes the encoded audio signal based on one piece of the priority information selected from the plurality of pieces of the priority information. 5. The decoding device according to claim 4, wherein the plurality of pieces of priority information are generated for each computational capability according to the computational capability of the decoding side of the encoded audio signal. The decoding device according to claim 1, further comprising a priority information generating section that generates the priority information based on the encoded audio signal. The decoding device according to claim 6, wherein the priority information generating section generates the priority information based on sound pressure or spectral shape of the audio signal obtained from the encoded audio signal. The audio signal decoding unit decodes, for each channel or each object, the predetermined time based on the priority information for the predetermined time and the priority information for the time before or after the predetermined time. 2. The decoding device according to claim 1, selecting whether to decode the encoded audio signal of. When the decoding is performed, the signal obtained by the decoding is used as an output signal, and when the decoding is not performed, 0 data is used as the output signal, and the output of the predetermined time is performed for each channel or each object. an addition unit that adds the signal and the output signal before or after the predetermined time to generate the audio signal of the predetermined time;
Adjusting the gain of the audio signal for the predetermined time based on the priority information for the predetermined time and the priority information for the time before or after the predetermined time for each channel or for each object. 2. The decoding device according to claim 1, further comprising: a gain adjustment unit that performs gain-adjusting a high-frequency power value for each channel or for each object based on the priority information for the predetermined time and the priority information for the time before or after the predetermined time; 10. The decoding device according to claim 9, further comprising a high frequency generator that generates high frequency components of the audio signal at the predetermined time based on the gain-adjusted power value and the audio signal at the predetermined time. . further comprising a high frequency generator that generates an audio signal of the predetermined time containing high frequency components based on the power value of the high frequency and the audio signal of the predetermined time for each channel or object;
10. The decoding device according to claim 9, wherein the gain adjustment section adjusts the gain of the audio signal for the predetermined time including high frequency components. 2. A rendering unit for generating audio signals for each of the plurality of channels by allocating an audio signal of the object to each of the plurality of channels with a predetermined gain value based on the priority information for the predetermined time. The decoding device according to . Obtaining encoded audio signals of multiple channels or multiple objects and priority information of each said audio signal at a given time;
decoding, based on the priority information, the encoded audio signal of a predetermined number of channels or objects according to the priority information. Obtaining encoded audio signals of multiple channels or multiple objects and priority information of each said audio signal at a given time;
A program that causes a computer to perform a process comprising, based on the priority information, decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information. a priority information generator for generating priority information at a given time of audio signals of multiple channels or multiple objects;
and a packing unit that stores the priority information in a bitstream. The encoding device according to claim 15, wherein the priority information generating section generates a plurality of pieces of the priority information for each audio signal. 17. The encoding device according to claim 16, wherein the priority information generation unit generates the priority information for each computational capability according to the computational capability of a decoding side of the encoded audio signal. 16. The encoding device according to claim 15, wherein the priority information generating section generates the priority information based on sound pressure or spectral shape of the audio signal. further comprising an encoding unit that encodes audio signals of the plurality of channels or the plurality of objects;
The encoding device according to claim 15, wherein the packing unit stores the priority information and the encoded audio signal in the bitstream. generating priority information at a given time for an audio signal of multiple channels or multiple objects;
An encoding method, comprising: storing the priority information in a bitstream. generating priority information at a given time for an audio signal of multiple channels or multiple objects;
A program that causes a computer to execute a process including a step of storing the priority information in a bitstream.

Images