JP2021064013A - Decoder and method, and program - Google Patents

Abstract

To enable the amount of calculation for decoding an audio signal to be reduced.SOLUTION: A priority information acquisition unit acquires priority information of each channel from a bit stream for supply to an output selection unit. A channel audio signal decoding unit decodes encoded data of an audio signal of each channel, and supplies an obtained MDCT coefficient to the output selection unit. The output selection unit supplies the MDCT coefficient to an IMDCT unit in the case where the priority degree indicated in the priority information is more than or equal to a predetermined degree, and supplies 0 as the MDCT coefficient to a 0-value output unit in the case where the priority degree is less than the predetermined degree. The 0-value output unit generates an audio signal on the basis of 0 supplied as the MDCT coefficient. In addition, the IMDCT unit performs IMDCT on the basis of the MDCT coefficient to generate an audio signal. The present technology can be applied to an encoder and a decoder.SELECTED DRAWING: Figure 10

Description

The present technology relates to decoding devices and methods and programs, and more particularly to decoding devices and methods and programs capable of reducing the computational complexity of decoding audio signals.

For example, as a method of encoding an audio signal, the international standards MPEG (Moving Picture Experts Group) -2 AAC (Advanced Audio Coding) standard, MPEG-4 AAC standard and MPEG-D USAC (Unified Speech and Audio Coding) Multi-channel coding of the standard is known (see, for example, Non-Patent Document 1 and Non-Patent Document 2).

INTERNATIONAL STANDARD ISO / IEC 14496-3 Fourth edition 2009-09-01 Information technology-coding of audio-visual objects-part3: Audio INTERNATIONAL STANDARD ISO / IEC 23003-3 Frist edition 2012-04-01 Information technology-coding of audio-visual objects-part3: Unified speech and audio coding

By the way, in order to reproduce more realistically than the conventional 5.1 channel surround reproduction and to transmit a plurality of sound materials (objects), a coding technique using more audio channels is required.

For example, consider a case where a 24-channel audio signal and an audio signal of a plurality of objects are encoded and decoded, and a case where a 2-channel audio signal is encoded and decoded. In such cases, it is possible to decode 2-channel audio signals in real time on mobile devices with poor computing power, but it is possible to decode 24-channel audio signals and audio signals of multiple objects in real time. It can be difficult.

With the current audio codecs such as MPEG-D USAC, it is necessary to decode the audio signals of all channels and all objects, so it is difficult to reduce the amount of calculation at the time of decoding. Then, depending on the device on the decoding side, it may not be possible to reproduce the audio signal in real time.

The present technology has been made in view of such a situation, and makes it possible to reduce the amount of decoding calculation.

The decoding device of one aspect of the present technology acquires the priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each of the objects from the supplied bit stream at a predetermined time. Based on the unit and the priority information, the encoded audio signal of the object whose priority indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority indicated in the priority information is determined. It includes an audio signal decoding unit that does not decode the encoded audio signal of the object that is less than the predetermined degree.

A decoding method or program of one aspect of the present technology obtains priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each object from a supplied bit stream at a predetermined time. Then, based on the priority information, the encoded audio signal of the object whose priority indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority indicated in the priority information is the said. The encoded audio signal of the object that is less than a predetermined degree includes a step of not decoding.

In one aspect of the present technology, priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each of the objects at a predetermined time is acquired from the supplied bit stream, and the priority is obtained. Based on the degree information, the encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority degree indicated in the priority information is less than the predetermined degree. The encoded audio signal of the object is not decoded.

It is a figure explaining a bit stream. It is a figure explaining the coding. It is a figure explaining the priority information. It is a figure explaining the meaning of the value of priority information. It is a figure which shows the structural example of the coding apparatus. It is a figure which shows the structural example of the channel audio coding part. It is a figure which shows the structural example of the object audio coding part. It is a flowchart explaining the coding process. It is a figure which shows the configuration example of the decoding apparatus. It is a figure which shows the structural example of the unpacking / decoding part. It is a flowchart explaining the decoding process. It is a flowchart explaining the selective decoding process. It is a figure which shows the other structural example of the unpacking / decoding part. It is a flowchart explaining the selective decoding process. It is a figure which shows an example of the syntax of the metadata of an object. It is a figure explaining the generation of an audio signal. It is a figure explaining the generation of an audio signal. It is a figure explaining the selection of the output destination of the MDCT coefficient. It is a figure explaining the gain adjustment of an audio signal and a power value of a high region. It is a figure explaining the gain adjustment of an audio signal and a power value of a high region. It is a figure which shows the other structural example of the unpacking / decoding part. It is a flowchart explaining the selective decoding process. It is a figure explaining the gain adjustment of an audio signal. It is a figure explaining the gain adjustment of an audio signal. It is a figure which shows the other structural example of the unpacking / decoding part. It is a flowchart explaining the selective decoding process. It is a figure explaining the VBAP gain. It is a figure explaining the VBAP gain. It is a figure which shows the other structural example of the unpacking / decoding part. It is a flowchart explaining the decoding process. It is a flowchart explaining the selective decoding process. It is a figure which shows the configuration example of a computer.

Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

<First Embodiment>
<Overview of this technology>
The present technology transmits the priority information of the audio signal of each channel and the priority information of the audio signal of each object in the coding of the audio signal of each channel constituting the multi-channel and the audio signal of the object. This is intended to reduce the amount of decoding calculation.

In addition, the present technology performs frequency-time conversion when the priority indicated in the priority information of each channel or each object is equal to or higher than a predetermined degree on the decoding side, and the priority indicated in the priority information is a predetermined degree. If it is less than, the frequency-time conversion is not performed and the result of the frequency-time conversion is set to 0 so that the amount of calculation for decoding the audio signal can be reduced.

In the following, a case where the multi-channel audio signal and the audio signal of the object are encoded according to the AAC standard will be described, but the same processing is performed when the multi-channel audio signal and the object audio signal are encoded by another method.

For example, when a multi-channel audio signal and an audio signal of a plurality of objects are encoded and transmitted according to the AAC standard, the audio signal of each channel or each object is encoded and transmitted frame by frame.

Specifically, as shown in FIG. 1, a coded audio signal and information necessary for decoding an audio signal are stored in a plurality of elements (bitstream elements), and a bitstream composed of these elements is transmitted. Will be done.

In this example, in the bit stream for one frame, t elements EL1 to element ELt are arranged in order from the beginning, and finally, an identifier TERM indicating that the end position is related to the information of the frame is arranged. ..

For example, the element EL1 arranged at the beginning is an ancillary data area called DSE (Data Stream Element), and information about each of a plurality of channels such as information about downmixing of audio signals and identification information is described in DSE. To.

A coded audio signal is stored in the element EL2 to the element ELt following the element EL1.

In particular, the element that stores the audio signal of a single channel is called SCE, and the element that stores the audio signal of two pairs of channels is called CPE. Also, the audio signal of each object is stored in SCE.

In the present technology, the priority information of the audio signals of each channel constituting the multi-channel and the priority information of the audio signals of each object are generated and stored in the DSE.

For example, it is assumed that the audio signals of consecutive frames F11 to F13 are encoded as shown in FIG.

In such a case, the encoder analyzes the priority of the audio signal of each channel for each frame, and the priority information of each channel is analyzed, for example, as shown in FIG. To generate. Similarly, the encoding device also generates priority information for the audio signal of each object.

For example, the coding device analyzes the priority of the audio signal based on the sound pressure of the audio signal, the shape of the spectrum, and the correlation of the spectrum shape between each channel or between objects.

In FIG. 3, priority information of each channel when the total number of channels is M channels is shown as an example. That is, for each channel from the channel having the channel number 0 to the channel having the channel number M-1, the numerical value indicating the priority degree of the audio signal of those channels is shown as the priority information.

For example, the priority information of the channel having the channel number 0 is 3, and the priority information of the channel having the channel number 1 is 0. Hereinafter, the channel having a predetermined channel number m (m = 0,1, ..., M-1) will also be referred to as a channel m.

The value of the priority information shown in FIG. 3 is set to any value from 0 to 7 as shown in FIG. 4, and the larger the value of the priority information, the more the audio signal is reproduced. It is said that the priority of, that is, the importance is high.

Therefore, the audio signal having the priority information value of 0 has the lowest priority, and the audio signal having the priority information value of 7 has the highest priority.

When a multi-channel audio signal or an audio signal of multiple objects is played back at the same time, the sound played by those audio signals usually includes sound that is less important than other sounds. In other words, even if a specific sound is not reproduced in the whole sound, there is a sound that does not cause discomfort to the listener.

Therefore, if it is necessary not to decode an audio signal having a low priority, it is possible to reduce the amount of decoding calculation while suppressing deterioration of sound quality. Therefore, in the encoding device, priority information indicating the degree of importance of each audio signal at the time of reproduction, that is, the degree of priority for decoding is provided for each frame so that the audio signal to be undecoded can be appropriately selected. Is given to each audio signal.

When the priority information of each audio signal is determined as described above, the priority information is stored in the DSE of the element EL1 shown in FIG. In particular, in the example of FIG. 3, since the number of channels constituting the multi-channel audio signal is M, the priority information of each of M channels from channel 0 to channel M-1 is stored in the DSE.

Similarly, the priority information of each object is also stored in the DSE of the element EL1. Here, for example, assuming that there are N objects having object numbers from 0 to N-1, priority information is determined for each of the N objects and stored in the DSE.

Hereinafter, an object having a predetermined object number n (n = 0,1, ..., N-1) will also be referred to as an object n.

If priority information is determined for each audio signal in this way, which audio signal is important at the time of reproduction on the reproduction side, that is, the decoding side of the audio signal, and which audio signal should be preferentially decoded, that is, used for reproduction. You can easily identify what should be done.

Returning to the description of FIG. 2, for example, it is assumed that the priority information of the audio signals of the frames F11 and the frame F13 of the predetermined channel is 7, and the priority information of the audio signals of the frame F12 of the predetermined channel is 0.

Further, it is assumed that the decoding side of the audio signal, that is, the decoding device (decoder) does not decode the audio signal having a priority lower than the predetermined priority.

Here, for example, assuming that a predetermined priority is called a threshold value and the threshold value is 4, in the above-described example, with respect to the audio signals of the frames F11 and the frame F13 of the predetermined channel whose priority information is 7. Is decrypted.

On the other hand, the audio signal of the frame F12 of the predetermined channel whose priority information is 0 is not decoded.

Therefore, in this example, the audio signal of the frame F12 is regarded as a silent signal, the audio signals of the frame F11 and the frame F13 are combined, and the audio signal of the final predetermined channel is obtained.

More specifically, for example, when each audio signal is encoded, time-frequency conversion is performed on the audio signal to encode the information obtained by the time-frequency conversion, and the resulting encoded data is stored in the element. To.

Although any processing may be performed as the time-frequency conversion, the description will be continued below assuming that MDCT (Modified Discrete Cosine Transform) (Modified Discrete Cosine Transform) is performed as the time-frequency conversion.

In the decoding device, the encoded data is decoded, and the MDCT coefficient obtained as a result is subjected to IMDCT (Inverse Modified Discrete Cosine Transform) (inverse modified discrete cosine transform) to generate an audio signal. .. That is, here, IMDCT is performed as the inverse conversion of the time-frequency conversion (frequency-time conversion).

Therefore, more specifically, IMDCT is performed on the frame F11 and the frame F13 whose priority information is the threshold value 4 or more, and an audio signal is generated.

Further, IMDCT is not performed for the frame F12 whose priority information is less than the threshold value 4, and the result of IMDCT is set to 0 to generate an audio signal. As a result, the audio signal of the frame F12 becomes a silent signal, that is, 0 data.

As yet another example, in the example shown in FIG. 3, when the threshold value is 4, among the audio signals of each channel 0 to channel M-1, the priority information is a value less than the threshold value 4. The audio signals of 0, channel 1, and channel M-2 will not be decoded.

As described above, the decoding calculation is performed while minimizing the deterioration of sound quality by not decoding the low priority audio signal indicated by the priority information according to the comparison result with the threshold value. The amount can be reduced.

<Configuration example of encoding device>
Next, specific embodiments of a coding device and a decoding device to which the present technology is applied will be described. First, the coding device will be described.

FIG. 5 is a diagram showing a configuration example of a coding device to which the present technology is applied.

The coding device 11 of FIG. 5 includes a channel audio coding unit 21, an object audio coding unit 22, a metadata input unit 23, and a packing unit 24.

The channel audio coding unit 21 is supplied with audio signals of each multi-channel channel having M channels. For example, the audio signal of each channel is supplied from the microphone corresponding to those channels. In FIG. 5, the characters "# 0" to "# M-1" represent channel numbers of each channel.

The channel audio coding unit 21 encodes the audio signal of each supplied channel, generates priority information based on the audio signal, and combines the coded data obtained by the coding and the priority information. It is supplied to the packing unit 24.

The object audio coding unit 22 is supplied with N audio signals of each object. For example, the audio signal for each object is supplied by a microphone attached to that object. In FIG. 5, the characters "# 0" to "# N-1" represent the object number of each object.

The object audio coding unit 22 encodes the audio signal of each supplied object, generates priority information based on the audio signal, and combines the coded data obtained by the coding and the priority information. It is supplied to the packing unit 24.

The metadata input unit 23 supplies the metadata of each object to the packing unit 24. For example, the metadata of an object is spatial position information indicating the position of the object in space. More specifically, for example, the spatial position information is three-dimensional coordinate information indicating the coordinates of the position of an object in the three-dimensional space.

The packing unit 24 is supplied from the coded data and priority information supplied from the channel audio coding unit 21, the coded data and priority information supplied from the object audio coding unit 22, and the metadata input unit 23. It packs the metadata to generate a bit stream and outputs it.

The bitstream thus obtained contains the coded data of each channel, the priority information of each channel, the coded data of each object, the priority information of each object, and the metadata of each object for each frame. It will be.

Here, the audio signals of the M channels and the audio signals of the N objects stored in the bit stream for one frame are audio signals of the same frame to be reproduced at the same time.

Here, an example in which priority information is generated for each audio signal for each frame as priority information for the audio signals of each channel or each object will be described, but with an arbitrary predetermined time as a unit. For example, one priority information may be generated for several frames of audio signals.

<Configuration example of channel audio encoding unit>
Further, the channel audio coding unit 21 of FIG. 5 is configured as shown in, for example, FIG. 6 in more detail.

The channel audio coding unit 21 shown in FIG. 6 includes a coding unit 51 and a priority information generation unit 52.

The coding unit 51 includes an MDCT unit 61, and the coding unit 51 encodes an audio signal of each channel supplied from the outside.

That is, the MDCT unit 61 performs MDCT on the audio signals of each channel supplied from the outside. The coding unit 51 encodes the MDCT coefficient of each channel obtained by MDCT, and supplies the coded data of each channel obtained as a result, that is, the coded audio signal to the packing unit 24.

Further, the priority information generation unit 52 analyzes the audio signals of each channel supplied from the outside, generates the priority information of the audio signals of each of those channels, and supplies the priority information to the packing unit 24.

<Configuration example of object audio encoding unit>
Further, the object audio coding unit 22 of FIG. 5 is configured as shown in FIG. 7, for example, in more detail.

The object audio coding unit 22 shown in FIG. 7 includes a coding unit 91 and a priority information generation unit 92.

The coding unit 91 includes an MDCT unit 101, and the coding unit 91 encodes an audio signal of each object supplied from the outside.

That is, the MDCT unit 101 performs MDCT on the audio signal of each object supplied from the outside. The coding unit 91 encodes the MDCT coefficient of each object obtained by MDCT, and supplies the coded data of each object obtained as a result, that is, the coded audio signal to the packing unit 24.

Further, the priority information generation unit 92 analyzes the audio signal of each object supplied from the outside, generates the priority information of the audio signal of each of the objects, and supplies the priority information to the packing unit 24.

<Explanation of coding process>
Next, the processing performed by the coding apparatus 11 will be described.

When the audio signals of the plurality of channels and the audio signals of the plurality of objects to be reproduced at the same time are supplied for one frame, the coding device 11 performs a coding process to obtain the coded audio signals. Output the included bitstream.

Hereinafter, the coding process by the coding apparatus 11 will be described with reference to the flowchart of FIG. Note that this coding process is performed for each frame of the audio signal.

In step S11, the priority information generation unit 52 of the channel audio coding unit 21 generates priority information of the audio signal of each supplied channel and supplies it to the packing unit 24. For example, the priority information generation unit 52 analyzes an audio signal for each channel and generates priority information based on the sound pressure and spectral shape of the audio signal, the correlation of the spectral shapes between channels, and the like.

In step S12, the packing unit 24 stores the priority information of the audio signal of each channel supplied from the priority information generation unit 52 in the DSE of the bit stream. That is, the priority information is stored in the first element of the bitstream.

In step S13, the priority information generation unit 92 of the object audio coding unit 22 generates the priority information of the audio signal of each supplied object and supplies it to the packing unit 24. For example, the priority information generation unit 92 analyzes an audio signal for each object and generates priority information based on the sound pressure and spectral shape of the audio signal, the correlation of the spectral shapes between the objects, and the like.

When generating the priority information of the audio signal of each channel or each object, the number of audio signals to which the priority is assigned for each priority that is the value of the priority information is set with respect to the number of channels and the number of objects. It may be predetermined.

For example, in the example of FIG. 3, the number of audio signals whose priority information is "7", that is, the number of channels is 5, the number of audio signals whose priority information is "6" is 3, and so on. It may be predetermined.

In step S14, the packing unit 24 stores the priority information of the audio signal of each object supplied from the priority information generation unit 92 in the DSE of the bit stream.

In step S15, the packing unit 24 stores the metadata of each object in the DSE of the bitstream.

For example, the metadata input unit 23 acquires the metadata of each object by receiving a user input operation, communicating with the outside, and reading from an external recording area, and the packing unit 24 Supply to. The packing unit 24 stores the metadata supplied from the metadata input unit 23 in the DSE in this way.

By the above processing, the DSE of the bitstream stores the priority information of the audio signals of all channels, the priority information of the audio signals of all objects, and the metadata of all objects.

In step S16, the coding unit 51 of the channel audio coding unit 21 encodes the audio signal of each supplied channel.

More specifically, the MDCT unit 61 performs MDCT on the audio signal of each channel, and the coding unit 51 encodes the MDCT coefficient of each channel obtained by MDCT, and the resulting channel The coded data is supplied to the packing unit 24.

In step S17, the packing unit 24 stores the coded data of the audio signal of each channel supplied from the coding unit 51 in the SCE or CPE of the bit stream. That is, the coded data is stored in each element arranged after the DSE in the bit stream.

In step S18, the coding unit 91 of the object audio coding unit 22 encodes the audio signal of each supplied object.

More specifically, the MDCT unit 101 performs MDCT on the audio signal of each object, and the coding unit 91 encodes the MDCT coefficient of each object obtained by MDCT, and the MDCT coefficient of each object obtained as a result is encoded. The coded data is supplied to the packing unit 24.

In step S19, the packing unit 24 stores the coded data of the audio signal of each object supplied from the coding unit 91 in the SCE of the bit stream. That is, the encoded data is stored in some elements arranged after the DSE in the bitstream.

By the above processing, the priority information and coded data of the audio signals of all channels, the priority information and coded data of the audio signals of all objects, and the metadata of all objects are stored for the frame to be processed. The resulting bitstream is obtained.

In step S20, the packing unit 24 outputs the obtained bit stream, and the coding process ends.

As described above, the coding device 11 generates the priority information of the audio signal of each channel and the priority information of the audio signal of each object, stores the information in the bit stream, and outputs the information. Therefore, the decoding side can easily grasp which audio signal has a higher priority.

As a result, the decoding side can selectively decode the encoded audio signal according to the priority information. As a result, it is possible to reduce the amount of decoding calculation while minimizing the deterioration of the sound quality of the sound reproduced by the audio signal.

In particular, by storing the priority information of the audio signal of each object in the bit stream, not only the amount of decoding calculation can be reduced on the decoding side, but also the amount of calculation for subsequent processing such as rendering can be reduced. it can.

<Configuration example of decoding device>
Next, a decoding device that takes the bit stream output from the coding device 11 described above as an input and decodes the coded data included in the bit stream will be described.

Such a decoding device is configured, for example, as shown in FIG.

The decoding device 151 shown in FIG. 9 includes an unpacking / decoding unit 161, a rendering unit 162, and a mixing unit 163.

The unpacking / decoding unit 161 acquires the bit stream output from the encoding device 11 and unpacks and decodes the bit stream.

The unpacking / decoding unit 161 supplies the audio signal of each object obtained by the unpacking and decoding and the metadata of each object to the rendering unit 162. At this time, the unpacking / decoding unit 161 decodes the coded data of each object according to the priority information included in the bit stream.

Further, the unpacking / decoding unit 161 supplies the audio signals of each channel obtained by the unpacking and decoding to the mixing unit 163. At this time, the unpacking / decoding unit 161 decodes the coded data of each channel according to the priority information included in the bit stream.

The rendering unit 162 generates an M channel audio signal based on the audio signal of each object supplied from the unpacking / decoding unit 161 and the spatial position information as metadata of each object, and supplies the audio signal to the mixing unit 163. To do. At this time, the rendering unit 162 generates M audio signals of each channel so that the sound image of each object is localized at the position indicated by the spatial position information of those objects.

The mixing unit 163 weights and adds the audio signal of each channel supplied from the unpacking / decoding unit 161 and the audio signal of each channel supplied from the rendering unit 162 for each channel, and finally each channel. Generates an audio signal. The mixing unit 163 supplies the final audio signal of each channel thus obtained to the speaker corresponding to each external channel, and reproduces the sound.

<Configuration example of unpacking / decoding unit>
Further, the unpacking / decoding unit 161 of the decoding device 151 shown in FIG. 9 is configured as shown in, for example, FIG. 10 in more detail.

The unpacking / decoding unit 161 shown in FIG. 10 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and an object audio unit. It has a signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a 0 value output unit 200, and an IMDCT unit 201.

The priority information acquisition unit 191 acquires the priority information of the audio signal of each channel from the supplied bit stream and supplies it to the output selection unit 194, and at the same time, obtains the priority information of the audio signal of each object from the bit stream. It is acquired and supplied to the output selection unit 199.

Further, the priority information acquisition unit 191 acquires the metadata of each object from the supplied bit stream and supplies it to the rendering unit 162, and supplies the bit stream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197. Supply.

The channel audio signal acquisition unit 192 acquires the coded data of each channel from the bit stream supplied from the priority information acquisition unit 191 and supplies the coded data to the channel audio signal decoding unit 193. The channel audio signal decoding unit 193 decodes the coded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the resulting MDCT coefficient to the output selection unit 194.

The output selection unit 194 selectively switches the output destination of the MDCT coefficient of each channel supplied from the channel audio signal decoding unit 193 based on the priority information of each channel supplied from the priority information acquisition unit 191.

That is, when the priority information for a predetermined channel is less than the predetermined threshold value P, the output selection unit 194 supplies the MDCT coefficient of the channel to the 0 value output unit 195 as 0. Further, when the priority information for a predetermined channel is equal to or higher than a predetermined threshold value P, the output selection unit 194 supplies the MDCT coefficient of the channel supplied from the channel audio signal decoding unit 193 to the IMDCT unit 196.

The 0-value output unit 195 generates an audio signal based on the MDCT coefficient supplied from the output selection unit 194, and supplies the audio signal to the mixing unit 163. In this case, the MDCT coefficient is 0, so a silent audio signal is generated.

The IMDCT unit 196 performs IMDCT based on the MDCT coefficient supplied from the output selection unit 194 to generate an audio signal, and supplies the audio signal to the mixing unit 163.

The object audio signal acquisition unit 197 acquires the coded data of each object from the bit stream supplied from the priority information acquisition unit 191 and supplies the coded data to the object audio signal decoding unit 198. The object audio signal decoding unit 198 decodes the coded data of each object supplied from the object audio signal acquisition unit 197, and supplies the resulting MDCT coefficient to the output selection unit 199.

The output selection unit 199 selectively switches the output destination of the MDCT coefficient of each object supplied from the object audio signal decoding unit 198 based on the priority information of each object supplied from the priority information acquisition unit 191.

That is, when the priority information for a predetermined object is less than the predetermined threshold value Q, the output selection unit 199 sets the MDCT coefficient of the object to 0 and supplies it to the 0 value output unit 200. Further, when the priority information for a predetermined object is equal to or higher than a predetermined threshold value Q, the output selection unit 199 supplies the MDCT coefficient of the object supplied from the object audio signal decoding unit 198 to the IMDCT unit 201.

The value of the threshold value Q may be the same as the value of the threshold value P, or may be a value different from the value of the threshold value P. By appropriately determining the threshold value P and the threshold value Q according to the computing power of the decoding device 151, the amount of calculation for decoding the audio signal is reduced to a calculation amount within the range in which the decoding device 151 can decode in real time. Can be made to.

The 0-value output unit 200 generates an audio signal based on the MDCT coefficient supplied from the output selection unit 199 and supplies it to the rendering unit 162. In this case, the MDCT coefficient is 0, so a silent audio signal is generated.

The IMDCT unit 201 performs IMDCT based on the MDCT coefficient supplied from the output selection unit 199 to generate an audio signal, and supplies the audio signal to the rendering unit 162.

<Explanation of decryption process>
Next, the operation of the decoding device 151 will be described.

When the decoding device 151 supplies a bit stream for one frame from the coding device 11, the decoding device 151 performs a decoding process to generate an audio signal and outputs the audio signal to the speaker. Hereinafter, the decoding process performed by the decoding device 151 will be described with reference to the flowchart of FIG.

In step S51, the unpacking / decoding unit 161 acquires the bit stream transmitted from the encoding device 11. That is, the bitstream is received.

In step S52, the unpacking / decoding unit 161 performs a selective decoding process.

The details of the selective decoding process will be described later, but in the selective decoding process, the coded data of each channel and the coded data of each object are selectively decoded based on the priority information. Then, the audio signal of each channel obtained as a result is supplied to the mixing unit 163, and the audio signal of each object is supplied to the rendering unit 162. Further, the metadata of each object acquired from the bit stream is supplied to the rendering unit 162.

In step S53, the rendering unit 162 renders the audio signal of each object based on the audio signal of each object supplied from the unpacking / decoding unit 161 and the spatial position information as metadata of each object.

For example, the rendering unit 162 generates an audio signal for each channel by VBAP (Vector Base Amplitude Pannning) based on the spatial position information so that the sound image of each object is localized at the position indicated by the spatial position information, and the mixing unit 163. Supply to.

In step S54, the mixing unit 163 weights and adds the audio signals of each channel supplied from the unpacking / decoding unit 161 and the audio signals of each channel supplied from the rendering unit 162 for each channel, and adds weight to the external speaker. Supply to. As a result, the audio signals of the channels corresponding to the speakers are supplied to each speaker, and each speaker reproduces the sound based on the supplied audio signals.

When the audio signal of each channel is supplied to the speaker, the decoding process ends.

As described above, the decoding device 151 acquires the priority information from the bit stream and decodes the coded data of each channel and each object according to the priority information.

<Explanation of selective decoding process>
Subsequently, the selective decoding process corresponding to the process of step S52 of FIG. 11 will be described with reference to the flowchart of FIG.

In step S81, the priority information acquisition unit 191 acquires the priority information of the audio signal of each channel and the priority information of the audio signal of each object from the supplied bit stream, and outputs the output selection unit 194 and the output selection unit 194, respectively. It is supplied to the output selection unit 199.

Further, the priority information acquisition unit 191 acquires the metadata of each object from the bit stream and supplies it to the rendering unit 162, and supplies the bit stream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197.

In step S82, the channel audio signal acquisition unit 192 sets and holds the channel number of the channel to be processed.

In step S83, the channel audio signal acquisition unit 192 determines whether or not the held channel number is less than the number of channels M.

If it is determined in step S83 that the channel number is less than M, in step S84, the channel audio signal decoding unit 193 decodes the encoded data of the audio signal of the channel to be processed.

That is, the channel audio signal acquisition unit 192 acquires the coded data of the channel to be processed from the bit stream supplied from the priority information acquisition unit 191 and supplies it to the channel audio signal decoding unit 193.

Then, the channel audio signal decoding unit 193 decodes the encoded data supplied from the channel audio signal acquisition unit 192, and supplies the MDCT coefficient obtained as a result to the output selection unit 194.

In step S85, the output selection unit 194 determines whether or not the priority information of the channel to be processed supplied from the priority information acquisition unit 191 is equal to or higher than the threshold value P specified by the upper control device or the like (not shown). To judge. Here, the threshold value P is determined according to, for example, the computing power of the decoding device 151.

When it is determined in step S85 that the priority information is equal to or higher than the threshold value P, the output selection unit 194 supplies the MDCT coefficient of the channel to be processed supplied from the channel audio signal decoding unit 193 to the IMDCT unit 196. , The process proceeds to step S86. In this case, since the priority of the audio signal of the channel to be processed is equal to or higher than the predetermined priority, decoding for that channel, more specifically, IMDCT is performed.

In step S86, the IMDCT unit 196 performs IMDCT based on the MDCT coefficient supplied from the output selection unit 194 to generate an audio signal of the channel to be processed and supplies it to the mixing unit 163. When the audio signal is generated, the process then proceeds to step S87.

On the other hand, when it is determined in step S85 that the priority information is less than the threshold value P, the output selection unit 194 supplies the 0 value output unit 195 with the MDCT coefficient set to 0.

The 0-value output unit 195 generates an audio signal of the channel to be processed from the MDCT coefficient of 0 supplied from the output selection unit 194, and supplies the audio signal to the mixing unit 163. Therefore, the 0-value output unit 195 does not substantially perform any processing for generating an audio signal such as IMDCT.

The audio signal generated by the 0-value output unit 195 is a silent signal. When the audio signal is generated, the process then proceeds to step S87.

If it is determined in step S85 that the priority information is less than the threshold value P, or if an audio signal is generated in step S86, the channel audio signal acquisition unit 192 sets 1 as the holding channel number in step S87. In addition, the channel number of the channel to be processed is updated.

When the channel number is updated, the process then returns to step S83, and the above-described process is repeated. That is, an audio signal of a new channel to be processed is generated.

If it is determined in step S83 that the channel number of the channel to be processed is not less than M, audio signals have been obtained for all channels, so the process proceeds to step S88.

In step S88, the object audio signal acquisition unit 197 sets the object number of the object to be processed to 0 and holds it.

In step S89, the object audio signal acquisition unit 197 determines whether or not the held object number is less than the number of objects N.

If it is determined in step S89 that the object number is less than N, in step S90, the object audio signal decoding unit 198 decodes the encoded data of the audio signal of the object to be processed.

That is, the object audio signal acquisition unit 197 acquires the coded data of the object to be processed from the bit stream supplied from the priority information acquisition unit 191 and supplies it to the object audio signal decoding unit 198.

Then, the object audio signal decoding unit 198 decodes the coded data supplied from the object audio signal acquisition unit 197, and supplies the MDCT coefficient obtained as a result to the output selection unit 199.

In step S91, the output selection unit 199 determines whether or not the priority information of the object to be processed supplied from the priority information acquisition unit 191 is equal to or higher than the threshold value Q specified by a higher-level control device or the like (not shown). To judge. Here, the threshold value Q is determined according to, for example, the computing power of the decoding device 151.

When it is determined in step S91 that the priority information is equal to or higher than the threshold value Q, the output selection unit 199 supplies the MDCT coefficient of the object to be processed supplied from the object audio signal decoding unit 198 to the IMDCT unit 201. , The process proceeds to step S92.

In step S92, the IMDCT unit 201 performs IMDCT based on the MDCT coefficient supplied from the output selection unit 199, generates an audio signal of the object to be processed, and supplies the audio signal to the rendering unit 162. When the audio signal is generated, the process then proceeds to step S93.

On the other hand, when it is determined in step S91 that the priority information is less than the threshold value Q, the output selection unit 199 supplies the 0 value output unit 200 with the MDCT coefficient set to 0.

The 0-value output unit 200 generates an audio signal of the object to be processed from the MDCT coefficient of 0 supplied from the output selection unit 199, and supplies the audio signal to the rendering unit 162. Therefore, the 0-value output unit 200 does not substantially perform any processing for generating an audio signal such as IMDCT.

The audio signal generated by the 0-value output unit 200 is a silent signal. When the audio signal is generated, the process then proceeds to step S93.

If it is determined in step S91 that the priority information is less than the threshold value Q, or if an audio signal is generated in step S92, the object audio signal acquisition unit 197 sets the object number held by 1 to 1 in step S93. In addition, the object number of the object to be processed is updated.

When the object number is updated, the process then returns to step S89, and the above-described process is repeated. That is, an audio signal of a new object to be processed is generated.

Further, in step S89, when it is determined that the object number of the object to be processed is not less than N, audio signals are obtained for all channels and objects, so that the selective decoding process is completed, and then the process is performed in FIG. Step S53.

As described above, whether the decoding device 151 compares the priority information and the threshold value for each channel or each object and decodes the audio signal encoded for each channel or object of the frame to be processed. Decoding the encoded audio signal while determining whether or not.

That is, in the decoding device 151, the encoded audio signals are decoded by a predetermined number according to the priority information of each audio signal, and the remaining audio signals are not decoded.

As a result, only the audio signal having a high priority can be selectively decoded according to the playback environment, and the amount of decoding calculation is reduced while minimizing the deterioration of the sound quality of the sound reproduced by the audio signal. be able to.

Moreover, by decoding the encoded audio signal based on the priority information of the audio signal of each object, not only the computational complexity of decoding the audio signal but also the subsequent processing such as the processing in the rendering unit 162 and the like. The amount of calculation of can be reduced.

<Modification 1 of the first embodiment>
<About priority information>
Although it has been described above that one priority information is generated for one audio signal of each channel or each object, a plurality of priority information may be generated.

In such a case, for example, each of the plurality of priority information is generated for each computing power according to the computational complexity of decoding, that is, the computing power of the decoding side.

Specifically, for example, priority information for a device having computing power equivalent to 2 channels is generated based on the amount of calculation for decoding an audio signal corresponding to 2 channels in real time.

In the priority information for such a device equivalent to two channels, for example, among all the audio signals, the priority is so that the audio signal having the lower priority, that is, the value close to 0 is assigned as the priority information is larger. Information is generated.

In addition, priority information for a device having a computing power equivalent to 24 channels is also generated based on, for example, the amount of calculation for decoding an audio signal corresponding to 24 channels in real time. In the priority information for a device equivalent to 24 channels, for example, the priority information is generated so that the number of audio signals having a higher priority, that is, a value close to 7 is assigned as the priority information among all audio signals. Will be done.

In this case, for example, in step S11 of FIG. 8, the priority information generation unit 52 generates priority information for the equipment corresponding to two channels for the audio signal of each channel, and 2 for the priority information. An identifier indicating that the device is for a channel-equivalent device is added and supplied to the packing unit 24.

Further, in step S11, the priority information generation unit 52 also generates priority information for the equipment corresponding to 24 channels for the audio signal of each channel, and also generates priority information for the equipment corresponding to 24 channels in the priority information. An identifier indicating that the information is for the purpose is added, and the information is supplied to the packing unit 24.

Similarly, in step S13 of FIG. 8, the priority information generation unit 92 also generates priority information for the equipment corresponding to 2 channels and priority information for the equipment corresponding to 24 channels and adds an identifier. , Supply to the packing unit 24.

As a result, it is possible to obtain a plurality of priority information according to the computing power of a playback device such as a portable audio player, a multifunctional mobile phone, a tablet computer, a television receiver, a personal computer, or a high-quality audio device. become.

For example, a playback device such as a portable audio player has a relatively low computing power. Therefore, in such a playback device, if the audio signal encoded based on the priority information for the device equivalent to two channels is decoded, the audio signal can be decoded. Audio signals can be played back in real time.

As described above, when a plurality of priority information is generated for one audio signal, the decoding device 151 uses, for example, a higher-level control device which of the plurality of priority information is used. The priority information acquisition unit 191 and the like are instructed whether to perform decoding. The instruction as to which priority information is used is given, for example, by supplying an identifier.

It should be noted that which identifier priority information is used may be predetermined for each decoding device 151.

In example priority information acquisition unit 191, when either using priority information in advance which identifiers have been established, or if the identifier is specified by the upper controller, in step S81 in FIG. 12, the priority information acquisition unit 191 acquires priority information to which a defined identifier is added. Then, the acquired priority information is supplied from the priority information acquisition unit 191 to the output selection unit 194 and the output selection unit 199.

In other words, from a plurality of priority information stored in the bit stream, one appropriate priority information is selected according to the computing power of the decoding device 151, more specifically, the unpacking / decoding unit 161 and the like. Will be done.

In this case, the priority information may be read from the bit stream by using different identifiers for the priority information of each channel and the priority information of each object.

In this way, by selecting and acquiring specific priority information from the plurality of priority information included in the bit stream, appropriate priority information can be obtained according to the computing power of the decoding device 151 and the like. You can select and decrypt. As a result, any decoding device 151 can reproduce the audio signal in real time.

<Second Embodiment>
<Configuration example of unpacking / decoding unit>
In the above, an example in which the bit stream output from the encoding device 11 contains priority information has been described, but depending on the coding device, the bit stream may not include the priority information. possible.

Therefore, the decoding device 151 may generate priority information. For example, priority information can be generated using information indicating the sound pressure of the audio signal and information indicating the spectral shape, which can be extracted from the encoded data of the audio signal included in the bit stream.

In this way, when the decoding device 151 generates the priority information, the unpacking / decoding unit 161 of the decoding device 151 is configured as shown in FIG. 13, for example. In FIG. 13, the same reference numerals are given to the portions corresponding to the cases in FIG. 10, and the description thereof will be omitted as appropriate.

The unpacking / decoding unit 161 shown in FIG. 13 includes a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, an object audio signal acquisition unit 197, and an object audio unit. It has a signal decoding unit 198, an output selection unit 199, a 0 value output unit 200, an IMDCT unit 201, a priority information generation unit 231 and a priority information generation unit 232.

The configuration of the unpacking / decoding unit 161 shown in FIG. 13 is that the priority information acquisition unit 191 is not provided, and the priority information generation unit 231 and the priority information generation unit 232 are newly provided. Unlike the unpacking / decoding unit 161 of FIG. 10, other configurations are the same as the unpacking / decoding unit 161 of FIG.

The channel audio signal acquisition unit 192 acquires the coded data of each channel from the supplied bit stream and supplies it to the channel audio signal decoding unit 193 and the priority information generation unit 231.

The priority information generation unit 231 generates priority information for each channel based on the coded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the priority information to the output selection unit 194.

The object audio signal acquisition unit 197 acquires the coded data of each object from the supplied bit stream and supplies it to the object audio signal decoding unit 198 and the priority information generation unit 232. Further, the object audio signal acquisition unit 197 acquires the metadata of each object from the supplied bit stream and supplies it to the rendering unit 162.

The priority information generation unit 232 generates priority information for each object based on the coded data of each object supplied from the object audio signal acquisition unit 197, and supplies the priority information to the output selection unit 199.

<Explanation of selective decoding process>
When the unpacking / decoding unit 161 has the configuration shown in FIG. 13, the decoding device 151 performs the selective decoding process shown in FIG. 14 as the process corresponding to step S52 of the decoding process shown in FIG. Hereinafter, the selective decoding process by the decoding device 151 will be described with reference to the flowchart of FIG.

In step S131, the priority information generation unit 231 generates priority information of the audio signal of each channel.

For example, the channel audio signal acquisition unit 192 acquires the coded data of each channel from the supplied bit stream and supplies it to the channel audio signal decoding unit 193 and the priority information generation unit 231.

The priority information generation unit 231 generates priority information for each channel based on the coded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the priority information to the output selection unit 194.

For example, the bitstream contains the scale factor, side information, and quantization spectrum for obtaining the MDCT coefficient as the coded data of the audio signal. Here, the scale factor is information indicating the sound pressure of the audio signal, and the quantization spectrum is information indicating the spectral shape of the audio signal.

The priority information generation unit 231 generates priority information of the audio signal of each channel based on the scale factor and the quantization spectrum included as the coded data of each channel. In this way, if the priority information is generated using the scale factor and the quantization spectrum, the priority information can be obtained immediately before decoding the coded data, and the calculation for generating the priority information can be performed. The amount can also be reduced.

In addition, the priority information is generated based on the sound pressure of the audio signal obtained by calculating the mean value of the square of the MDCT coefficient and the spectral shape of the audio signal obtained from the peak wrapping of the MDCT coefficient. It may be. In this case, the priority information generation unit 231 decodes the encoded data as appropriate, or acquires the MDCT coefficient from the channel audio signal decoding unit 193.

When the priority information of each channel is obtained, the processes of steps S132 to S137 are subsequently performed, but since these processes are the same as the processes of steps S82 to S87 of FIG. 12, the description thereof will be omitted. .. However, in this case, since the coded data of each channel has already been acquired, only the coded data is decoded in step S134.

If it is determined in step S133 that the channel number is not less than M, the priority information generation unit 232 generates priority information of the audio signal of each object in step S138.

For example, the object audio signal acquisition unit 197 acquires the coded data of each object from the supplied bit stream and supplies it to the object audio signal decoding unit 198 and the priority information generation unit 232. Further, the object audio signal acquisition unit 197 acquires the metadata of each object from the supplied bit stream and supplies it to the rendering unit 162.

The priority information generation unit 232 generates priority information for each object based on the coded data of each object supplied from the object audio signal acquisition unit 197, and supplies the priority information to the output selection unit 199. For example, priority information is generated based on scale factors and quantization spectra as in each channel.

Further, priority information may be generated based on the sound pressure and the spectral shape obtained from the MDCT coefficient. In this case, the priority information generation unit 232 decodes the encoded data as appropriate and acquires the MDCT coefficient from the object audio signal decoding unit 198.

When the priority information of each object is obtained, the processes of steps S139 to S144 are performed thereafter to end the selective decoding process, but these processes are the same as the processes of steps S88 to S93 of FIG. Therefore, the description thereof will be omitted. However, in this case, since the coded data of each object has already been acquired, only the coded data is decoded in step S141.

When the selective decoding process is completed, the process then proceeds to step S53 of FIG.

As described above, the decoding device 151 generates priority information of the audio signal of each channel or each object based on the coded data included in the bit stream. By generating the priority information in the decoding device 151 in this way, it is possible to obtain appropriate priority information for each audio signal with a small amount of calculation, and it is possible to reduce the amount of calculation for decoding and the amount of calculation for rendering. it can. In addition, deterioration of the sound quality of the sound reproduced by the audio signal can be minimized.

The priority information acquisition unit 191 of the unpacking / decoding unit 161 shown in FIG. 10 tried to acquire the priority information of the audio signals of each channel and each object from the supplied bit stream, but the bit stream Priority information may be generated when the priority information cannot be obtained from. In such a case, the priority information acquisition unit 191 performs the same processing as the priority information generation unit 231 and the priority information generation unit 232, and obtains the priority information of the audio signals of each channel and each object from the coded data. Generate.

<Third embodiment>
<About the threshold of priority information>
Further, in the above, it has been explained that for each channel and each object, the audio signal to be decoded by comparing the priority information with the threshold value P and the threshold value Q, and more specifically, the MDCT coefficient for performing IMDCT is selected. The threshold value P and the threshold value Q of the audio signal may be dynamically changed for each frame of the audio signal.

For example, the priority information acquisition unit 191 of the unpacking / decoding unit 161 shown in FIG. 10 can acquire priority information of each channel and each object from the bit stream without requiring decoding.

Therefore, for example, if the priority information acquisition unit 191 reads the priority information of the audio signals of all channels, the distribution of the priority information in the frame to be processed can be obtained. Further, the decoding device 151 knows in advance its own computing power, such as how many channels can be processed at the same time, that is, in real time.

Therefore, the priority information acquisition unit 191 determines the threshold value P of the priority information for the frame to be processed based on the distribution of the priority information in the frame to be processed and the computing power of the decoding device 151. May be good.

For example, the threshold value P is set so that the largest number of audio signals can be decoded within the range in which the decoding device 151 can perform processing in real time.

Further, the priority information acquisition unit 191 can dynamically determine the threshold value Q as in the case of the threshold value P. In this case, the priority information acquisition unit 191 obtains the distribution of the priority information based on the priority information of the audio signals of all the objects, and processes based on the obtained distribution and the computing power of the decoding device 151. A threshold Q of priority information for the target frame is set.

Such determination of the threshold value P and the threshold value Q can be performed with a relatively small amount of calculation.

By dynamically changing the threshold value of the priority information in this way, it is possible to minimize the deterioration of the sound quality of the sound reproduced by the audio signal while performing decoding in real time. In particular, in such a case, it is not necessary to prepare a plurality of priority information and it is not necessary to provide an identifier in the priority information, so that the code amount of the bit stream can be reduced.

<About object metadata>
Further, in the first to third embodiments described above, it has been described that the first element of the bitstream stores the metadata and priority information of the object for one frame. ..

In this case, the syntax of the part of the first element of the bitstream in which the metadata and priority information of the object is stored is as shown in FIG. 15, for example.

In the example shown in FIG. 15, the spatial position information and the priority information of the object are stored in the metadata of the object for one frame.

In this example, "num_objects" indicates the number of objects. In addition, "object_priority [o]" indicates the priority information of the O-th object. Here, the O-th object is an object specified by an object number.

“Position_azimuth [o]” indicates a horizontal angle representing the three-dimensional spatial position of the O-th object as seen from the user who is the viewer, that is, as viewed from a predetermined reference position. Further, "position_elevation [o]" indicates a vertical angle representing the three-dimensional spatial position of the O-th object as seen from the user who is the viewer. Furthermore, "position_radius [o]" indicates the distance from the viewer to the Oth object.

Therefore, the position of the object in the three-dimensional space is specified from these "position_azimuth [o]", "position_elevation [o]", and "position_radius [o]", and this information is the spatial position of the object. It is regarded as information.

Also, "gain_factor [o]" indicates the gain of the Oth object.

Thus, the metadata shown in FIG. 15 includes "object_priority [o]", "position_azimuth [o]", "position_elevation [o]", "position_radius [o]", and "gain_factor [" for one object. o] ”are arranged in order as the data of the object. Then, in the metadata, the data of each object is arranged and arranged in the order of the object numbers of the objects, for example.

<Fourth Embodiment>
<About noise caused by complete reconstruction and discontinuity of audio signals>
In the above, when the priority information of each frame (hereinafter, particularly referred to as a time frame) for each channel or object read from the bit stream by the decoding device 151 is less than a predetermined threshold value, IMDCT or the like. An example of reducing the processing amount at the time of decoding was described by omitting the decoding process of. Specifically, it has been described that when the priority information is less than the threshold value, the 0 value output unit 195 and the 0 value output unit 200 output a silent audio signal, that is, 0 data is output as an audio signal.

However, in such a case, the sound quality deteriorates in terms of hearing. Specifically, sound quality deterioration due to complete reconstruction of the audio signal and sound quality deterioration due to generation of noise due to signal discontinuity such as glitch noise occur.

(Sound quality deterioration due to complete reconstruction)
For example, if 0 data is output as an audio signal when the priority information is less than the threshold value, sound quality deterioration occurs when switching between the output of 0 data and the output of a normal audio signal that is not 0 data.

As described above, in the unpacking / decoding unit 161, the IMDCT unit 196 and the IMDCT unit 201 perform IMDCT with respect to the MDCT coefficient for each time frame read from the bit stream. Then, in more detail, the unpacking / decoding unit 161 generates an audio signal of the current time frame from the result or 0 data of the IMDCT about the current time frame and the result or 0 data of the IMDCT one hour before. To.

Here, the generation of the audio signal will be described with reference to FIG. Here, the generation of the audio signal of the object will be described as an example, but the same applies to the generation of the audio signal of each channel. Further, in the following, the audio signal output from the 0 value output unit 200 and the audio signal output from the IMDCT unit 201 will also be referred to as an IMDCT signal in particular. Similarly, the audio signal output from the 0-value output unit 195 and the audio signal output from the IMDCT unit 196 are also referred to as an IMDCT signal in particular.

In FIG. 16, in the figure, the horizontal direction indicates the time, and the rectangles with the characters “data [n-1]” to “data [n + 2]” are the time frames (n) of the predetermined objects. Represents a bitstream from -1) to time frame (n + 2). Further, the numerical value in the bit stream of each time frame indicates the value of the priority information of the object of that time frame, and in this example, the value of the priority information of each time frame is "7".

Further, in FIG. 16, the rectangles marked with the letters “MDCT_coef [q]” (where q = n-1, n, ...) represent the MDCT coefficients of the time frame (q), respectively.

Now, assuming that the threshold value Q = 4, the value "7" of the priority information in the time frame (n-1) is equal to or higher than the threshold value Q. Therefore, the IMDCT with respect to the MDCT coefficient for the time frame (n-1). Is done. Similarly, since the value "7" of the priority information in the time frame (n) is also equal to or higher than the threshold value Q, IMDCT is performed on the MDCT coefficient for the time frame (n).

As a result, it is assumed that the IMDCT signal OPS11 in the time frame (n-1) and the IMDCT signal OPS12 in the time frame (n) are obtained.

In this case, the unpacking / decoding unit 161 adds the first half of the IMDCT signal OPS12 of the time frame (n) and the second half of the IMDCT signal OPS11 of the time frame (n-1) one hour before. It is an audio signal of a time frame (n), that is, an audio signal of a period FL (n). In other words, the period FL (n) part of the IMDCT signal OPS11 and the period FL (n) part of the IMDCT signal OPS12 are overlapped and added, and the time frame (n) before encoding of the object to be processed is added. Audio signal is reproduced.

Such processing is necessary for the IMDCT signal to be completely reconstructed into the signal before MDCT.

However, in the unpacking / decoding unit 161 described above, at the timing of switching between the IMDCT signal of the IMDCT unit 201 and the IMDCT signal of the 0 value output unit 200 according to the priority information of each time frame, for example, as shown in FIG. , The IMDCT signal is no longer completely reconstructed into the pre-MDCT signal. That is, if 0 data is used instead of the original signal at the time of overlap addition, the original audio signal cannot be reproduced because it is not completely reconstructed, and the audible sound quality of the audio signal deteriorates. ..

In FIG. 17, the same characters and the like are written in the parts corresponding to the case in FIG. 16, and the description thereof will be omitted.

In the example of FIG. 17, the value of the priority information of the time frame (n-1) is "7", but the priority information of the other time frames (n) to the time frame (n + 2) is the lowest "7". It is "0".

Therefore, assuming that the threshold value Q = 4, for the time frame (n-1), the IMDCT unit 201 performs IMDCT with respect to the MDCT coefficient, and the IMDCT signal OPS21 of the time frame (n-1) is obtained. On the other hand, for the time frame (n), the IMDCT for the MDCT coefficient is not performed, and the 0 data output from the 0 value output unit 200 is used as the IMDCT signal OPS22 for the time frame (n).

In this case, the first half of 0 data, which is the IMDCT signal OPS22 of the time frame (n), and the second half of the IMDCT signal OPS21 of the time frame (n-1) one hour before that are added together to make the final It is regarded as an audio signal of the time frame (n). That is, the parts of the period FL (n) of the IMDCT signal OPS22 and the IMDCT signal OPS21 are overlapped and added to obtain the audio signal of the final time frame (n) of the object to be processed.

When the output source of the IMDCT signal is switched from the IMDCT unit 201 to the 0 value output unit 200 or from the 0 value output unit 200 to the IMDCT unit 201 in this way, the IMDCT signal from the IMDCT unit 201 is not completely reconstructed. Deterioration of audible sound quality occurs.

(Sound quality deterioration due to noise caused by discontinuity)
Further, when the output source of the IMDCT signal is switched from the IMDCT unit 201 to the 0 value output unit 200, or from the 0 value output unit 200 to the IMDCT unit 201, the signal is not completely reconstructed, so that the IMDCT signal obtained by the IMDCT is not completely reconstructed. And, the signal may be discontinuous at the connection part with the IMDCT signal which is set to 0 data. Then, glitch noise is generated in the discontinuous connection portion, and the audible sound quality of the audio signal is deteriorated.

Further, in order to improve the sound quality in the unpacking / decoding unit 161, SBR (Spectral Band) is applied to the audio signal obtained by overlapping and adding the IMDCT signals output from the IMDCT unit 201 and the 0 value output unit 200. Replication) etc. may be performed.

Various processes can be considered as the subsequent processes of the IMDCT unit 201 and the 0 value output unit 200, but the description will be continued below by taking SBR as an example.

In SBR, the high-frequency component of the original audio signal before encoding is generated from the audio signal obtained by overlap addition, which is a low-frequency component, and the high-frequency power value stored in the bitstream. Will be done.

Specifically, the audio signal for one hour frame is divided into several sections called time slots, and the audio signal in each time slot is a signal of a plurality of low-frequency subbands (hereinafter, low-frequency subband signal). The band is divided into (also called).

Then, a signal of each high-frequency subband (hereinafter, also referred to as a high-frequency subband signal) is generated based on the low-frequency subband signal of each subband and the power value of each subband on the high-frequency side. .. For example, the target high-frequency sub-band signal is generated by power-adjusting or frequency-shifting the low-frequency sub-band signal of a predetermined sub-band according to the power value of the target sub-band in the high frequency range. ..

Furthermore, the high-frequency subband signal and the low-frequency subband signal are combined to generate an audio signal containing a high-frequency component, and the audio signal including the high-frequency component generated for each time slot is combined to generate a high-frequency component. It is an audio signal of 1 hour frame containing components.

When such SBR is performed in the subsequent stage of the IMDCT unit 201 and the 0 value output unit 200, a high frequency component is generated by the SBR for the audio signal composed of the IMDCT signal output from the IMDCT unit 201. However, since the IMDCT signal output from the 0-value output unit 200 is 0 data, the high-frequency component obtained by SBR is also 0 data for the audio signal consisting of the IMDCT signal output from the 0-value output unit 200. It ends up.

Then, when the output source of the IMDCT signal is switched from the IMDCT unit 201 to the 0 value output unit 200, or from the 0 value output unit 200 to the IMDCT unit 201, the connection portion becomes discontinuous even in the high frequency range. There is. In such a case, glitch noise is generated and the audible sound quality is deteriorated.

Therefore, in this technology, the output destination of the MDCT coefficient is selected in consideration of the time frames before and after, and the fade-in processing and the fade-out processing for the audio signal are performed to suppress the above-mentioned deterioration of the audible sound quality and improve the sound quality. I did.

<Selection of MDCT coefficient output destination considering the time frame before and after>
First, the selection of the output destination of the MDCT coefficient in consideration of the time frames before and after will be described. Although the audio signal of the object will be described as an example here as well, the same applies to the audio signal of each channel. In addition, the processing described below is performed for each object and each channel.

For example, in the above-described embodiment, the output selection unit 199 has described that the output destination of the MDCT coefficient of each object is selectively switched based on the priority information of the current time frame. On the other hand, in the present embodiment, the output selection unit 199 is temporally continuous with the current time frame, the time frame immediately before the current time frame, and the time frame immediately after the current time frame. The output destination of the MDCT coefficient is switched based on the priority information of one time frame. In other words, whether or not to decode the encoded data is selected based on the priority information of three consecutive time frames.

Specifically, the output selection unit 199 supplies the MDCT coefficient of the time frame (n) of the object to the IMDCT unit 201 when the conditional expression shown in the following equation (1) is satisfied for the object to be processed.

In the equation (1), object_priority [q] (where q = n-1, n, n + 1) indicates the priority information of each time frame (q), and thre indicates the threshold value Q.

Therefore, if there is a time frame in which the priority information is equal to or higher than the threshold value Q in a total of three consecutive time frames, that is, the current time frame and the time frames before and after the current time frame, the MDCT coefficient is supplied as the supply destination. The IMDCT unit 201 is selected. In this case, the encoded data is decoded, and more specifically, the IMDCT for the MDCT coefficient is performed. On the other hand, when the priority information of those three time frames is all less than the threshold value Q, the MDCT coefficient is set to 0 and output to the 0 value output unit 200. In this case, decoding of the coded data, more specifically IMDCT on the MDCT coefficient, is not substantially performed.

As a result, as shown in FIG. 18, the audio signal is completely reconstructed from the IMDCT signal, and the deterioration of the audible sound quality is suppressed. In FIG. 18, the same characters and the like are written in the parts corresponding to the case in FIG. 16, and the description thereof will be omitted.

In the example shown on the upper side of FIG. 18, the value of the priority information of each time frame is the same as the example shown in FIG. For example, assuming that the threshold value Q = 4, in the example shown on the upper side in the figure, the priority information of the time frame (n-1) is equal to or higher than the threshold value Q, but the time frame (n) to the time frame (n + 2). In, the priority information is less than the threshold value Q.

Therefore, from the conditional expression shown in the equation (1), IMDCT is performed on the MDCT coefficients of the time frame (n-1) and the time frame (n), and the IMDCT signal OPS31 and the IMDCT signal OPS32 are obtained, respectively. On the other hand, in the time frame (n + 1) in which the conditional expression is not satisfied, IMDCT for the MDCT coefficient is not performed, and 0 data is set as the IMDCT signal OPS33.

Therefore, the audio signal of the time frame (n) that was not completely reconstructed in the example of FIG. 17 is completely reconstructed in the example shown on the upper side of FIG. 18, and the deterioration of the audible sound quality is suppressed. .. However, in this example, since the audio signal is not completely reconstructed in the next time frame (n + 1), the fade-out process described later is performed in the time frame (n) and the time frame (n + 1), which is audibly audible. Deterioration of sound quality is suppressed.

Further, in the example shown on the lower side in the figure, the priority information is less than the threshold value Q in the time frame (n-1) to the time frame (n + 1), and the priority is given in the time frame (n + 2). The information is equal to or higher than the threshold Q.

Therefore, from the conditional expression shown in the equation (1), the IMDCT for the MDCT coefficient is not performed in the time frame (n) when the conditional expression is not satisfied, and 0 data is regarded as the IMDCT signal OPS41. On the other hand, IMDCT is performed on the MDCT coefficients of the time frame (n + 1) and the time frame (n + 2), and the IMDCT signal OPS42 and the IMDCT signal OPS43 are obtained, respectively.

In this example, the audio signal can be completely reconstructed in the time frame (n + 2) when the priority information is switched from the value less than the threshold value Q to the value above the threshold value Q, so that the audible sound quality is deteriorated. Can be suppressed. However, even in this case, since the audio signal is not completely reconstructed in the time frame (n + 1) immediately before that, the fade-in processing described later is performed in the time frame (n + 1) and the time frame (n + 2). Therefore, the deterioration of the audible sound quality is suppressed.

Here, the priority information is pre-read for one hour frame, and the output destination of the MDCT coefficient is selected from the priority information of consecutive three-hour frames. Therefore, the fade-out process is performed in the time frame (n) and the time frame (n + 1) of the example shown on the upper side in the figure, and the time frame (n + 1) and the time of the example shown on the lower side in the figure. Fade-in processing is performed at the frame (n + 2).

However, if the priority information for 2 hour frames can be pre-read, the fade-out process is performed in the time frame (n + 1) and the time frame (n + 2) in the example shown on the upper side in the figure. Therefore, the fade-in process may be performed in the time frame (n) and the time frame (n + 1) of the example shown at the lower side in the figure.

<About fade-in processing and fade-out processing>
Next, the fade-in process and the fade-out process for the audio signal will be described. Although the audio signal of the object will be described as an example here as well, the same applies to the audio signal of each channel. Further, the fade-in process and the fade-out process are performed for each object and each channel.

In the present technology, for example, as in the example shown in FIG. 18, the time frame in which the IMDCT signal obtained by IMDCT and the IMDCT signal which is 0 data are overlapped and added and the time frame before or after the fade-in Processing or fading out processing is performed.

In the fade-in process, the gain of the audio signal is adjusted so that the amplitude (magnitude) of the audio signal in the time frame increases with time. Conversely, in the fade-out process, the gain of the audio signal is adjusted so that the amplitude of the audio signal in that time frame decreases with time.

As a result, deterioration of audible sound quality can be suppressed even when the connection portion between the IMDCT signal obtained by IMDCT and the IMDCT signal set to 0 data is discontinuous. Hereinafter, the gain value multiplied by the audio signal at the time of such gain adjustment will also be referred to as a fading signal gain in particular.

Further, in the present technology, fade-in processing or fade-out processing is also performed in SBR for the connection portion between the IMDCT signal obtained by IMDCT and the IMDCT signal which is 0 data.

That is, in SBR, the power value of each high-frequency subband is used for each time slot, but in this technology, the gain value determined for each time slot for fade-in processing or fade-out processing is set for each high-frequency band. SBR is performed by multiplying the power value of the subband. That is, the gain of the high frequency power value is adjusted.

Hereinafter, the gain value determined for each time slot, which is multiplied by the power value in the high frequency range, is also referred to as a fading SBR gain.

Specifically, the fading SBR gain for fade-in processing is set so that the gain value increases with time, that is, the fading SBR gain of the time slot rearward in time increases. Has been done. On the contrary, the fading SBR gain for the fade-out process is set so that the value becomes smaller as the fading SBR gain in the time slot rearward in time.

In this way, by performing the fade-in process and the fade-out process even during SBR, it is possible to suppress the deterioration of the audible sound quality even when the high frequencies are discontinuous.

Specifically, for example, the processes shown in FIGS. 19 and 20 are performed as gain adjustments such as fade-in processing and fade-out processing for the audio signal and the power value in the high frequency range. Note that, in FIGS. 19 and 20, the same characters, symbols, and the like are written in the parts corresponding to the cases in FIG. 18, and the description thereof will be omitted.

The example shown in FIG. 19 is an example of the case shown on the upper side in the figure of FIG. In this example, the time frame (n) and time frame (n + 1) audio signals are multiplied by the fading signal gain shown on the polygonal line GN11.

The value of the fading signal gain shown on the polygonal line GN11 changes linearly from "1" to "0" with time in the time frame (n) part, and continues to be "" in the time frame (n + 1) part. It is "0". Therefore, by adjusting the gain of the audio signal by the fading signal gain, the audio signal gradually changes to 0 data, so that deterioration of the audible sound quality can be suppressed.

Further, in this example, the fading SBR gain indicated by the arrow GN12 is multiplied by the high-frequency power value of each time slot in the time frame (n).

The value of the fading SBR gain indicated by the arrow GN12 changes from "1" to "0" so as to become smaller in time toward the rear time slot. Therefore, by adjusting the high-frequency gain by the fading SBR gain, the high-frequency component of the audio signal gradually changes to 0 data, so that deterioration of audible sound quality can be suppressed.

On the other hand, the example shown in FIG. 20 is an example shown on the lower side in the figure of FIG. In this example, the time frame (n + 1) and time frame (n + 2) audio signals are multiplied by the fading signal gain shown on the polygonal line GN21.

The value of the fading signal gain shown on the polygonal line GN21 is continuously "0" in the time frame (n + 1) part, and starts from "0" with time in the time frame (n + 2) part. It changes linearly up to "1". Therefore, by adjusting the gain of the audio signal by the fading signal gain, the audio signal gradually changes from 0 data to the original signal, so that deterioration of audible sound quality can be suppressed.

Further, in this example, the fading SBR gain indicated by the arrow GN22 is multiplied by the high-frequency power value of each time slot in the time frame (n + 2).

The value of the fading SBR gain indicated by the arrow GN22 changes from "0" to "1" so as to increase in time toward the rear time slot. Therefore, by adjusting the high-frequency gain by the fading SBR gain, the high-frequency component of the audio signal gradually changes from 0 data to the original signal, so that deterioration of audible sound quality can be suppressed.

<Configuration example of unpacking / decoding unit>
When the selection of the output destination of the MDCT coefficient described above and the gain adjustment such as the fade-in process and the fade-out process are performed, the unpacking / decoding unit 161 is configured as shown in FIG. 21, for example. In FIG. 21, the same reference numerals are given to the portions corresponding to the cases in FIG. 10, and the description thereof will be omitted as appropriate.

The unpacking / decoding unit 161 shown in FIG. 21 includes a priority information acquisition unit 191 and a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and overlap addition. Unit 271, Gain adjustment unit 272, SBR processing unit 273, Object audio signal acquisition unit 197, Object audio signal decoding unit 198, Output selection unit 199, 0 value output unit 200, IMDCT unit 201, Overlap addition unit 274, Gain adjustment It is composed of a unit 275 and an SBR processing unit 276.

The configuration of the unpacking / decoding unit 161 shown in FIG. 21 is such that the overlap adding unit 271 to the SBR processing unit 276 are further provided in addition to the configuration of the unpacking / decoding unit 161 shown in FIG.

The overlap addition unit 271 generates an audio signal for each time frame by overlapping addition of the IMDCT signal (audio signal) supplied from the 0 value output unit 195 or the IMDCT unit 196, and supplies the audio signal to the gain adjustment unit 272. To do.

The gain adjustment unit 272 gain-adjusts the audio signal supplied from the overlap addition unit 271 based on the priority information supplied from the priority information acquisition unit 191 and supplies the audio signal to the SBR processing unit 273.

The SBR processing unit 273 acquires the power value of each high-frequency subband for each time slot from the priority information acquisition unit 191 and obtains the high-frequency power value based on the priority information supplied from the priority information acquisition unit 191. Adjust the gain of the power value. Further, the SBR processing unit 273 performs SBR on the audio signal supplied from the gain adjustment unit 272 using the gain-adjusted high-frequency power value, and the audio signal obtained as a result is transmitted to the mixing unit 163. Supply.

The overlap addition unit 274 generates an audio signal for each time frame by overlapping addition of the IMDCT signal (audio signal) supplied from the 0 value output unit 200 or the IMDCT unit 201, and supplies the audio signal to the gain adjustment unit 275. To do.

The gain adjustment unit 275 gain-adjusts the audio signal supplied from the overlap addition unit 274 based on the priority information supplied from the priority information acquisition unit 191 and supplies the audio signal to the SBR processing unit 276.

The SBR processing unit 276 acquires the power value of each high-frequency subband for each time slot from the priority information acquisition unit 191 and obtains the high-frequency power value based on the priority information supplied from the priority information acquisition unit 191. Adjust the gain of the power value. Further, the SBR processing unit 276 performs SBR on the audio signal supplied from the gain adjustment unit 275 using the gain-adjusted high-frequency power value, and transmits the resulting audio signal to the rendering unit 162. Supply.

<Explanation of selective decoding process>
Subsequently, the operation of the decoding device 151 when the unpacking / decoding unit 161 has the configuration shown in FIG. 21 will be described. In this case, the decoding device 151 performs the decoding process described with reference to FIG. However, as the selective decoding process in step S52, the process shown in FIG. 22 is performed.

Hereinafter, the selective decoding process corresponding to the process of step S52 of FIG. 11 will be described with reference to the flowchart of FIG. 22.

In step S181, the priority information acquisition unit 191 acquires the high-frequency power value of the audio signal of each channel from the supplied bitstream and supplies it to the SBR processing unit 273, and from the bitstream, of each object. The high-frequency power value of the audio signal is acquired and supplied to the SBR processing unit 276.

When the high frequency power value is acquired, the processing of steps S182 to S187 is performed thereafter to generate an audio signal (IMDCT signal) of the channel to be processed. These processings are performed in steps S81 to S81 to FIG. Since it is the same as the process of step S86, the description thereof will be omitted.

However, in step S186, when the same conditional expression as the above-mentioned equation (1) is satisfied, that is, the priority information of the current time frame of the channel to be processed, and each time frame immediately before and after the current time frame. If even one of the priority information is equal to or higher than the threshold value P, it is determined that the priority information is equal to or higher than the threshold value P. Further, the IMDCT signal generated by the 0 value output unit 195 or the IMDCT unit 196 is output to the overlap addition unit 271.

If it is not determined in step S186 that the priority information is equal to or higher than the threshold value P, or if an IMDCT signal is generated in step S187, the process of step S188 is performed.

In step S188, the overlap addition unit 271 performs overlap addition of the IMDCT signal supplied from the 0 value output unit 195 or the IMDCT unit 196, and outputs the audio signal of the current time frame obtained as a result to the gain adjustment unit 272. Supply.

Specifically, for example, as described with reference to FIG. 18, the first half of the IMDCT signal of the current time frame and the second half of the IMDCT signal of the immediately preceding time frame are added to obtain the audio signal of the current time frame. Will be done.

In step S189, the gain adjustment unit 272 gain-adjusts the audio signal supplied from the overlap addition unit 271 based on the priority information of the channel to be processed supplied from the priority information acquisition unit 191 and performs SBR processing. Supply to unit 273.

Specifically, in the gain adjustment unit 272, the priority information of the time frame immediately before the current time frame is equal to or higher than the threshold value P, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame are given. When the information is less than the threshold P, the gain of the audio signal is adjusted by the fading signal gain shown by the polygonal line GN11 in FIG. In this case, the time frame (n) in FIG. 19 corresponds to the current time frame, and in the time frame immediately after the current time frame, the gain is adjusted with the fading signal gain = 0 as shown by the polygonal line GN11. ..

Further, when the priority information of the current time frame is equal to or higher than the threshold value P and the priority information of the 2-hour frame immediately before the current time frame is both less than the threshold value P, the gain adjusting unit 272 indicates the break line GN21 in FIG. Adjust the gain of the audio signal with the faded signal gain shown. In this case, the time frame (n + 2) in FIG. 20 corresponds to the current time frame, and in the time frame immediately before the current time frame, the gain adjustment at the fading signal gain = 0 is performed as shown by the polygonal line GN21. Will be done.

The gain adjusting unit 272 adjusts the gain only in these two examples, does not adjust the gain in other cases, and supplies the audio signal to the SBR processing unit 273 as it is.

In step S190, the SBR processing unit 273 transfers the audio signal supplied from the gain adjusting unit 272 to the audio signal supplied from the gain adjusting unit 272 based on the high frequency power value and priority information of the channel to be processed, which is supplied from the priority information acquisition unit 191. Perform SBR for it.

Specifically, in the SBR processing unit 273, the priority information of the time frame immediately before the current time frame is equal to or higher than the threshold value P, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame are prioritized. When the degree information is less than the threshold value P, the gain of the high frequency power value is adjusted by the fading SBR gain shown by the arrow GN12 in FIG. That is, the fading SBR gain is multiplied by the high frequency power value.

Then, the SBR processing unit 273 performs SBR using the gain-adjusted high-frequency power value, and supplies the audio signal obtained as a result to the mixing unit 163. In this case, the time frame (n) in FIG. 19 corresponds to the current time frame.

Further, when the priority information of the current time frame is equal to or higher than the threshold value P and the priority information of the 2-hour frame immediately before the current time frame is both lower than the threshold value P, the SBR processing unit 273 indicates the arrow GN 22 in FIG. Adjust the high frequency power value with the indicated fading SBR gain. Then, the SBR processing unit 273 performs SBR using the gain-adjusted high-frequency power value, and supplies the audio signal obtained as a result to the mixing unit 163. In this case, the time frame (n + 2) in FIG. 20 corresponds to the current time frame.

The SBR processing unit 273 adjusts the gain of the high frequency power value only in these two examples, and does not adjust the gain in other cases, and keeps the acquired high frequency power value as it is. SBR is performed using this, and the audio signal obtained as a result is supplied to the mixing unit 163.

When the SBR is performed and the audio signal of the current time frame is obtained, the processes of steps S191 to S196 are subsequently performed, but these processes are the same as the processes of steps S87 to S92 of FIG. The description thereof will be omitted.

However, in step S195, when the conditional expression of the above-mentioned equation (1) is satisfied, it is determined that the priority information is equal to or higher than the threshold value Q. Further, the IMDCT signal (audio signal) generated by the 0 value output unit 200 or the IMDCT unit 201 is output to the overlap addition unit 274.

When the IMDCT signal of the current time frame is obtained in this way, the processes of steps S197 to S199 are performed to generate the audio signal of the current time frame, and these processes are the same as the processes of steps S188 to S190. Since it is the same, the description thereof will be omitted.

When the object audio signal acquisition unit 197 adds 1 to the object number in step S200, the process returns to step S193. Then, if it is determined in step S193 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S53 of FIG.

As described above, the unpacking / decoding unit 161 selects the output destination of the MDCT coefficient according to the priority information of the current time frame and the time frames before and after the current time frame. As a result, the audio signal is completely reconstructed at the switching portion between the time frame in which the priority information is equal to or higher than the threshold value and the time frame in which the priority information is less than the threshold value, thereby suppressing deterioration of the audible sound quality. can do.

Further, the unpacking / decoding unit 161 gain-adjusts the audio signal after overlap addition and the power value in the high frequency band based on the priority information of consecutive 3-hour frames. That is, fade-in processing and fade-out processing are performed as appropriate. As a result, it is possible to suppress the generation of glitch noise and suppress the deterioration of the audible sound quality.

<Fifth Embodiment>
<About fade-in processing and fade-out processing>
In the fourth embodiment, it has been described that the gain is adjusted for the audio signal after the overlap addition, and the gain is adjusted for the high frequency power value at the time of SBR. In this case, gain adjustment, that is, fade-in processing and fade-out processing are performed separately for the low-frequency component and the high-frequency component of the final audio signal.

Therefore, in order to realize these fade-in processing and fade-out processing with less processing, gain adjustment is performed on the audio signal obtained by SBR without performing gain adjustment immediately after overlap addition and during SBR. You may do so.

In such a case, for example, the gain adjustment is performed as shown in FIGS. 23 and 24. In addition, in FIG. 23 and FIG. 24, the same characters and the like are written in the parts corresponding to the cases in FIGS. 19 and 20, and the description thereof will be omitted.

The example shown in FIG. 23 is an example in which the change in priority information is the same as the case shown in FIG. In this example, assuming that the threshold value Q = 4, the priority information of the time frame (n-1) is equal to or higher than the threshold value Q, but the priority information in the time frame (n) to the time frame (n + 2). Is less than the threshold Q.

In such a case, the audio signal obtained by SBR in the time frame (n) and the time frame (n + 1) is multiplied by the fading signal gain shown by the polygonal line GN31 to adjust the gain. Become.

The fading signal gain shown by the polygonal line GN31 is the same as the fading signal gain shown by the polygonal line GN11 in FIG. However, in the case of the example of FIG. 23, since the audio signal to be gain-adjusted includes both low-frequency components and high-frequency components, the gains of those low-frequency components and high-frequency components are included. Adjustments can be made with a single fading signal gain.

By adjusting the gain of the audio signal by such a fading signal gain, the audio signal is gradually added to the portion where the IMDCT signal obtained by IMDCT and the IMDCT signal set as 0 data are overlapped and added, and the portion immediately before that. It will change to 0 data. As a result, deterioration of sound quality in terms of hearing can be suppressed.

On the other hand, the example shown in FIG. 24 is an example in which the change in priority information is the same as the case shown in FIG. In this example, assuming that the threshold value Q = 4, the priority information in the time frame (n) and the time frame (n + 1) is less than the threshold value Q, but the priority information in the time frame (n + 2) is It is equal to or higher than the threshold value Q.

In such a case, the audio signal obtained by SBR in the time frame (n + 1) and the time frame (n + 2) is multiplied by the fading signal gain shown by the polygonal line GN41 to adjust the gain. It will be.

The fading signal gain shown by the polygonal line GN41 is the same as the fading signal gain shown by the polygonal line GN21 of FIG. However, in the case of the example of FIG. 24, since the audio signal to be gain-adjusted includes both low-frequency components and high-frequency components, the gains of those low-frequency components and high-frequency components are included. Adjustments can be made with a single fading signal gain.

By adjusting the gain of the audio signal by such a fading signal gain, the audio signal is 0 in the part where the IMDCT signal obtained by IMDCT and the IMDCT signal which is regarded as 0 data are overlapped and added and the part immediately after that. It will gradually change from the data to the original signal. As a result, deterioration of sound quality in terms of hearing can be suppressed.

<Configuration example of unpacking / decoding unit>
When the gain adjustment by the fade-in process or the fade-out process described with reference to FIGS. 23 and 24 is performed, the unpacking / decoding unit 161 is configured as shown in FIG. 25, for example. In FIG. 25, the same reference numerals are given to the portions corresponding to the cases in FIG. 21, and the description thereof will be omitted as appropriate.

The unpacking / decoding unit 161 shown in FIG. 25 includes a priority information acquisition unit 191 and a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and overlap addition. Unit 271, SBR processing unit 273, gain adjustment unit 272, object audio signal acquisition unit 197, object audio signal decoding unit 198, output selection unit 199, 0 value output unit 200, IMDCT unit 201, overlap addition unit 274, SBR processing. It is composed of a unit 276 and a gain adjusting unit 275.

The configuration of the unpacking / decoding unit 161 shown in FIG. 25 is shown in FIG. 21 in that the gain adjusting unit 272 and the gain adjusting unit 275 are arranged after the SBR processing unit 273 and the SBR processing unit 276, respectively. The configuration is different from that of the unpacking / decoding unit 161.

In the unpacking / decoding unit 161 shown in FIG. 25, the SBR processing unit 273 with respect to the audio signal supplied from the overlap addition unit 271 based on the high frequency power value supplied from the priority information acquisition unit 191. SBR is performed, and the audio signal obtained as a result is supplied to the gain adjustment unit 272. In this case, the SBR processing unit 273 does not adjust the gain of the high frequency power value.

The gain adjusting unit 272 gain-adjusts the audio signal supplied from the SBR processing unit 273 based on the priority information supplied from the priority information acquisition unit 191 and supplies it to the mixing unit 163.

The SBR processing unit 276 performs SBR on the audio signal supplied from the overlap addition unit 274 based on the high-frequency power value supplied from the priority information acquisition unit 191 and obtains the audio signal as a result. Is supplied to the gain adjusting unit 275. In this case, the SBR processing unit 276 does not adjust the gain of the high frequency power value.

The gain adjusting unit 275 gain-adjusts the audio signal supplied from the SBR processing unit 276 based on the priority information supplied from the priority information acquisition unit 191 and supplies the audio signal to the rendering unit 162.

<Explanation of selective decoding process>
Subsequently, the operation of the decoding device 151 when the unpacking / decoding unit 161 has the configuration shown in FIG. 25 will be described. In this case, the decoding device 151 performs the decoding process described with reference to FIG. However, as the selective decoding process in step S52, the process shown in FIG. 26 is performed.

Hereinafter, the selective decoding process corresponding to the process of step S52 of FIG. 11 will be described with reference to the flowchart of FIG. 26. Since the processing of steps S231 to S238 is the same as the processing of steps S181 to S188 of FIG. 22, the description thereof will be omitted. However, in step S232, priority information is not supplied to the SBR processing unit 273 and the SBR processing unit 276.

In step S239, the SBR processing unit 273 performs SBR on the audio signal supplied from the overlap addition unit 271 based on the high frequency power value supplied from the priority information acquisition unit 191 and obtains the result. The generated audio signal is supplied to the gain adjustment unit 272.

In step S240, the gain adjusting unit 272 gain-adjusts the audio signal supplied from the SBR processing unit 273 based on the priority information of the channel to be processed supplied from the priority information acquisition unit 191 and adjusts the gain to the mixing unit 163. Supply to.

Specifically, in the gain adjustment unit 272, the priority information of the time frame immediately before the current time frame is equal to or higher than the threshold value P, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame are given. When the information is less than the threshold P, the gain of the audio signal is adjusted by the fading signal gain shown by the polygonal line GN31 in FIG. In this case, the time frame (n) in FIG. 23 corresponds to the current time frame, and in the time frame immediately after the current time frame, the gain is adjusted with the fading signal gain = 0 as shown by the polygonal line GN31. ..

Further, when the priority information of the current time frame is equal to or higher than the threshold value P and the priority information of the 2-hour frame immediately before the current time frame is both less than the threshold value P, the gain adjusting unit 272 indicates the break line GN41 in FIG. Adjust the gain of the audio signal with the faded signal gain shown. In this case, the time frame (n + 2) in FIG. 24 corresponds to the current time frame, and in the time frame immediately before the current time frame, the gain adjustment at the fading signal gain = 0 is performed as shown by the polygonal line GN41. Will be done.

The gain adjusting unit 272 adjusts the gain only in these two examples, does not adjust the gain in other cases, and supplies the audio signal to the mixing unit 163 as it is.

After the gain adjustment of the audio signal is performed, the processes of steps S241 to S247 are performed thereafter, but these processes are the same as the processes of steps S191 to S197 of FIG. 22, and the description thereof will be omitted.

When the audio signal of the current time frame of the object to be processed is obtained in this way, the processing of steps S248 and S249 is performed to generate the final audio signal of the current time frame. Since it is the same as the processing of step S239 and step S240, the description thereof will be omitted.

When the object audio signal acquisition unit 197 adds 1 to the object number in step S250, the process returns to step S243. Then, if it is determined in step S243 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S53 of FIG.

As described above, the unpacking / decoding unit 161 gain-adjusts the audio signal obtained by SBR based on the priority information of consecutive 3-hour frames. As a result, it is possible to more easily suppress the generation of glitch noise and suppress the deterioration of the audible sound quality.

In this embodiment, an example of selecting the output destination of the MDCT coefficient using the priority information for 3 hours frames and adjusting the gain by the fading signal gain has been described, but the gain by the fading signal gain has been described. Only adjustments may be made.

In such a case, the output selection unit 194 and the output selection unit 199 select the output destination of the MDCT coefficient by the same processing as in the case of the first embodiment. Then, in the gain adjusting unit 272 and the gain adjusting unit 275, when the priority information of the current time frame is less than the threshold value, the fading signal gain of the current time frame is linearly increased or decreased to perform fade-in processing or fade-out. Perform processing. Here, whether the fade-in process or the fade-out process is performed may be determined from the priority information of the current time frame and the priority information of the time frames before and after that.

<Sixth Embodiment>
<About fade-in processing and fade-out processing>
By the way, in the rendering unit 162, for example, VBAP is performed and the audio signal of each channel for reproducing the sound of each object is generated from the audio signal of each object.

Specifically, in VBAP, the gain value of the audio signal (hereinafter, also referred to as VBAP gain) is calculated for each time frame for each object, that is, for each channel, that is, for each speaker that outputs audio. Then, the sum of the audio signals of each object multiplied by the VBAP gain for the same channel (speaker) is taken as the audio signal of that channel. In other words, for each object, the object's audio signal is assigned to each of those channels with the VBAP gain calculated for each channel.

Therefore, for the audio signal of the object, instead of adjusting the gain of the audio signal of the object and the power value in the high frequency range, the VBAP gain is adjusted appropriately to suppress the generation of glitch noise and the audible sound quality. You may try to suppress the deterioration of.

In such a case, for example, linear interpolation is performed on the VBAP gain of each time frame, the VBAP gain of each sample of the audio signal in each time frame is calculated, and the obtained VBAP gain is used to calculate the audio signal of each channel. Is generated.

For example, the value of the VBAP gain of the first sample of the time frame to be processed is the value of the VBAP gain of the sample at the end of the time frame immediately before the time frame to be processed. Further, the value of the VBAP gain of the sample at the end of the time frame to be processed is the value of the VBAP gain calculated by the normal VBAP for the time frame to be processed.

Then, in the time frame to be processed, the value of the VBAP gain of each sample between the first sample and the last sample is determined so that the VBAP gain changes linearly from the first sample to the last sample.

However, if the priority information of the time frame to be processed is less than the threshold value, the VBAP is not calculated, and the value of the VBAP gain of the sample at the end of the time frame to be processed is set to 0. Then, the VBAP gain of each sample is determined so that the VBAP gain changes linearly from the first sample of the time frame to be processed to the last sample.

By adjusting the gain of the audio signal of each object by the VBAP gain in this way, the gain of the low-frequency component and the high-frequency component can be adjusted at once, and the generation of glitch noise is suppressed with a smaller amount of processing. However, it is possible to suppress deterioration of sound quality in terms of hearing.

When the VBAP gain is determined for each sample in this way, the VBAP gain for each sample in each time frame is shown in FIGS. 27 and 28, for example.

In addition, in FIG. 27 and FIG. 28, the same characters and the like are written in the parts corresponding to the cases in FIGS. 19 and 20, and the description thereof will be omitted. Further, in FIGS. 27 and 28, “VBAP_gain [q] [s]” (where q = n-1, n, n + 1, n + 2) is a speaker that identifies a speaker corresponding to a predetermined channel. It shows the VBAP gain of the time frame (q) of the object to be processed, which has an index of s.

The example shown in FIG. 27 is an example in which the change in priority information is the same as the case shown in FIG. In this example, assuming that the threshold value Q = 4, the priority information of the time frame (n-1) is equal to or higher than the threshold value Q, but the priority information in the time frame (n) to the time frame (n + 2). Is less than the threshold Q.

In such a case, the VBAP gain of the time frame (n-1) to the time frame (n + 1) is, for example, the gain shown by the polygonal line GN51.

In this example, since the priority information of the time frame (n-1) is equal to or higher than the threshold value Q, the VBAP gain of each sample is determined based on the VBAP gain calculated by the normal VBAP.

That is, the value of the VBAP gain of the sample at the beginning of the time frame (n-1) is the same as the value of the VBAP gain of the sample at the end of the time frame (n-2). Further, the value of the VBAP gain of the sample at the end of the time frame (n-1) corresponds to the speaker s calculated by the normal VBAP for the time frame (n-1) for the object to be processed. It is the value of the VBAP gain of the channel. The VBAP gain value of each sample in the time frame (n-1) is set to change linearly from the first sample to the last sample.

Further, since the priority information of the time frame (n) is less than the threshold value Q, the value of the VBAP gain of the sample at the end of the time frame (n) is set to 0.

That is, the value of the VBAP gain of the sample at the beginning of the time frame (n) is the same as the value of the VBAP gain of the sample at the end of the time frame (n-1), and the VBAP of the sample at the end of the time frame (n). The value of the gain is set to 0. Then, the value of the VBAP gain of each sample in the time frame (n) is determined so as to change linearly from the first sample to the last sample.

Furthermore, since the priority information of the time frame (n + 1) is less than the threshold Q, the value of the VBAP gain of the sample at the end of the time frame (n + 1) is set to 0, and as a result, the time frame (n + 1) The value of VBAP gain of all samples in) is 0.

In this way, by setting the value of the VBAP gain of the sample at the end of the time frame in which the priority information is less than the threshold value Q to 0, the fade-out process equivalent to the example of FIG. 23 becomes possible.

On the other hand, the example shown in FIG. 28 is an example in which the change in priority information is the same as the case shown in FIG. 24. In this example, assuming that the threshold value Q = 4, the priority information is less than the threshold value Q in the time frame (n-1) to the time frame (n + 1), but the priority of the time frame (n + 2). The information is equal to or higher than the threshold value Q.

In such a case, the VBAP gain of the time frame (n-1) to the time frame (n + 2) is, for example, the gain shown by the polygonal line GN61.

In this example, both the priority information of the time frame (n) and the priority information of the time frame (n + 1) are less than the threshold value Q, so that the VBAP gain of all the samples in the time frame (n + 1) is 0. Become.

Further, since the priority information of the time frame (n + 2) is equal to or higher than the threshold value Q, the object to be processed is based on the VBAP gain of the channel corresponding to the speaker s calculated by the normal VBAP. The VBAP gain of each sample is determined.

That is, the value of the VBAP gain of the first sample of the time frame (n + 2) is set to 0, which is the value of the VBAP gain of the last sample of the time frame (n + 1), and the value of the time frame (n + 2). The value of the VBAP gain of the sample at the end is the value of the VBAP gain calculated by the normal VBAP for the time frame (n + 2). The VBAP gain value of each sample in the time frame (n + 2) is set to change linearly from the first sample to the last sample.

In this way, by setting the value of the VBAP gain of the sample at the end of the time frame in which the priority information is less than the threshold value Q to 0, the fade-in processing equivalent to the example of FIG. 24 becomes possible.

<Configuration example of unpacking / decoding unit>
When the gain adjustment by the fade-in process or the fade-out process described with reference to FIGS. 27 and 28 is performed, the unpacking / decoding unit 161 is configured as shown in FIG. 29, for example. In FIG. 29, the same reference numerals are given to the portions corresponding to the cases in FIG. 25, and the description thereof will be omitted as appropriate.

The unpacking / decoding unit 161 shown in FIG. 29 includes a priority information acquisition unit 191 and a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a 0 value output unit 195, an IMDCT unit 196, and overlap addition. Section 271, SBR processing section 273, gain adjustment section 272, object audio signal acquisition section 197, object audio signal decoding section 198, output selection section 199, 0 value output section 200, IMDCT section 201, overlap addition section 274, and SBR. It is composed of a processing unit 276.

The configuration of the unpacking / decoding unit 161 shown in FIG. 29 is different from the configuration of the unpacking / decoding unit 161 shown in FIG. 25 in that the gain adjusting unit 275 is not provided, and is otherwise the same. ing.

In the unpacking / decoding unit 161 shown in FIG. 29, the SBR processing unit 276 refers to the audio signal supplied from the overlap addition unit 274 based on the high frequency power value supplied from the priority information acquisition unit 191. SBR is performed, and the audio signal obtained as a result is supplied to the rendering unit 162.

Further, the priority information acquisition unit 191 acquires the metadata and priority information of each object from the supplied bit stream and supplies the metadata to the rendering unit 162. The priority information of each object is also supplied to the output selection unit 199.

<Explanation of decryption process>
Subsequently, the operation of the decoding device 151 when the unpacking / decoding unit 161 has the configuration shown in FIG. 29 will be described.

In this case, the decoding device 151 performs the decoding process shown in FIG. Hereinafter, the decoding process performed by the decoding device 151 will be described with reference to the flowchart of FIG. However, in step S281, the same processing as that of step S51 in FIG. 11 is performed, and thus the description thereof will be omitted.

In step S282, the unpacking / decoding unit 161 performs a selective decoding process.

Here, the selective decoding process corresponding to the process of step S282 of FIG. 30 will be described with reference to the flowchart of FIG. 31.

Since the processing of steps S311 to S328 is the same as the processing of steps S231 to S248 of FIG. 26, the description thereof will be omitted. However, in step S312, the priority information acquisition unit 191 also supplies the priority information acquired from the bit stream to the rendering unit 162.

In step S329, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S323. Then, if it is determined in step S323 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S283 of FIG.

Therefore, in the selective decoding process shown in FIG. 31, the gain of the audio signal of each channel is adjusted by the fading signal gain as in the case of the fifth embodiment, and the gain is adjusted for each object. The audio signal obtained by SBR is output to the rendering unit 162 as it is without being performed.

Returning to the description of the decoding process of FIG. 30, in step S283, the rendering unit 162 uses the audio signal of each object supplied from the SBR processing unit 276 and the metadata of each object supplied from the priority information acquisition unit 191. The audio signal of each object is rendered based on the position information of the object and the priority information of the current time frame of each object.

For example, as described with reference to FIGS. 27 and 28, the rendering unit 162 provides priority information of the current time frame for each channel and a sample VBAP at the end of the time frame immediately before the current time frame for each object. Based on the gain, calculate the VBAP gain for each sample in the current time frame. At this time, the rendering unit 162 appropriately calculates the VBAP gain by VBAP based on the position information.

Then, the rendering unit 162 generates an audio signal of each channel based on the VBAP gain for each sample of each channel calculated for each object and the audio signal of each object, and supplies the audio signal to the mixing unit 163.

Although an example of calculating the VBAP gain of each sample so that the VBAP gain of each sample in the time frame changes linearly has been described here, the VBAP gain may change non-linearly. Further, although an example in which the audio signal of each channel is generated by VBAP has been described, even when the audio signal of each channel is generated by another method, the gain of the audio signal of each object is obtained by the same processing as in the case of VBAP. It is possible to adjust.

When the audio signal of each channel is generated, the process of step S284 is performed and the decoding process is terminated. However, since the process of step S284 is the same as the process of step S54 of FIG. 11, the description thereof is omitted. To do.

In this way, the decoding device 151 calculates the VBAP gain for each sample based on the priority information for each object, and adjusts the gain of the object's audio signal by the VBAP gain when the audio signal of each channel is generated. As a result, it is possible to suppress the generation of glitch noise with a smaller amount of processing and suppress the deterioration of the audible sound quality.

In the fourth to sixth embodiments, the output destination of the MDCT coefficient is selected by using the priority information of the time frames immediately before and after the current time frame, the fading signal gain, etc. It was explained that the gain is adjusted by. However, not limited to this, the priority information of the current time frame, the priority information of the time frame before the predetermined time frame of the current time frame, and the priority information of the time frame after the predetermined time frame of the current time frame And may be used.

By the way, the series of processes described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software are installed on the computer. Here, the computer includes a computer embedded in dedicated hardware and, for example, a general-purpose computer capable of executing various functions by installing various programs.

FIG. 32 is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes programmatically.

In a computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.

An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, and the like. The output unit 507 includes a display, a speaker, and the like. The recording unit 508 includes a hard disk, a non-volatile memory, and the like. The communication unit 509 includes a network interface and the like. The drive 510 drives a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer configured as described above, the CPU 501 loads the program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executes the above-described series. Is processed.

The program executed by the computer (CPU 501) can be recorded and provided on the removable media 511 as a package media or the like, for example. Programs can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting.

In the computer, the program can be installed in the recording unit 508 via the input / output interface 505 by mounting the removable media 511 in the drive 510. Further, the program can be received by the communication unit 509 and installed in the recording unit 508 via a wired or wireless transmission medium. In addition, the program can be pre-installed in the ROM 502 or the recording unit 508.

The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be a program that is processed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

Further, the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

For example, the present technology can have a cloud computing configuration in which one function is shared by a plurality of devices via a network and jointly processed.

Further, each step described in the above-mentioned flowchart can be executed by one device or can be shared and executed by a plurality of devices.

Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.

Further, the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

Further, the present technology can also have the following configurations.

(1)
An acquisition unit that acquires encoded audio signals of a plurality of channels or a plurality of objects, and priority information of each said audio signal at a predetermined time.
A decoding device including an audio signal decoding unit that decodes the encoded audio signal of a predetermined number of channels or objects according to the priority information based on the priority information.
(2)
The decoding device according to (1), wherein the audio signal decoding unit decodes the encoded audio signal whose priority degree indicated by the priority information is equal to or higher than a predetermined degree.
(3)
The decoding device according to (2), wherein the acquisition unit changes the predetermined degree based on the priority information of the audio signals of the plurality of channels or the plurality of objects at the predetermined time.
(4)
The acquisition unit acquires a plurality of the priority information for each audio signal, and obtains the plurality of priority information.
The audio signal decoding unit decodes the encoded audio signal based on one of the priority information selected from the plurality of priority information (1) to (3). The decoding device according to paragraph 1.
(5)
The decoding device according to (4), wherein the plurality of priority information is generated for each of the computing powers according to the computing power of the decoding side of the coded audio signal.
(6)
The decoding device according to any one of (1) to (5), further comprising a priority information generation unit that generates the priority information based on the encoded audio signal.
(7)
The decoding device according to (6), wherein the priority information generation unit generates the priority information based on the sound pressure or spectral shape of the audio signal obtained from the encoded audio signal.
(8)
The audio signal decoding unit determines the predetermined time based on the priority information of the predetermined time and the priority information of the time before or after the predetermined time for each channel or object. The decoding device according to (1), wherein the encoded audio signal of the above is selected.
(9)
When the decoding is performed, the signal obtained by the decoding is used as an output signal, and when the decoding is not performed, 0 data is used as an output signal, and the output for the predetermined time is used for each channel or object. An adder that adds the signal and the output signal at a time before or after the predetermined time to generate an audio signal at the predetermined time.
For each channel or object, gain adjustment of the audio signal at the predetermined time is performed based on the priority information at the predetermined time and the priority information at a time before or after the predetermined time. The decoding device according to (1), further comprising a gain adjusting unit for performing the operation.
(10)
For each channel or object, the gain of the high frequency power value is adjusted based on the priority information at the predetermined time and the priority information at a time before or after the predetermined time, and the gain is adjusted. The decoding device according to (9), further comprising a high frequency generation unit that generates a high frequency component of the audio signal of the predetermined time based on the gain-adjusted power value and the audio signal of the predetermined time. ..
(11)
For each channel or object, a high frequency generator is further provided to generate an audio signal of the predetermined time including a high frequency component based on the power value of the high frequency band and the audio signal of the predetermined time.
The decoding device according to (9), wherein the gain adjusting unit adjusts the gain of an audio signal containing a high frequency component for the predetermined time.
(12)
A rendering unit is further provided which allocates an audio signal of an object to each of a plurality of channels at a predetermined gain value based on the priority information of the predetermined time and generates an audio signal of the plurality of channels (1). Decoding device according to.
(13)
The encoded audio signals of a plurality of channels or a plurality of objects, and the priority information of each said audio signal at a predetermined time are acquired.
A decoding method comprising decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information based on the priority information.
(14)
The encoded audio signals of a plurality of channels or a plurality of objects, and the priority information of each said audio signal at a predetermined time are acquired.
A program that causes a computer to perform a process including decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information based on the priority information.
(15)
A priority information generator that generates priority information for audio signals of multiple channels or multiple objects at a predetermined time, and
A coding device including a packing unit for storing the priority information in a bit stream.
(16)
The coding device according to (15), wherein the priority information generation unit generates a plurality of the priority information for each audio signal.
(17)
The coding device according to (16), wherein the priority information generation unit generates the priority information for each of the computing powers according to the computing power of the decoding side of the coded audio signal.
(18)
The coding device according to any one of (15) to (17), wherein the priority information generation unit generates the priority information based on the sound pressure or the spectral shape of the audio signal.
(19)
Further comprising a coding unit for encoding the audio signals of the plurality of channels or the plurality of objects.
The coding device according to any one of (15) to (18), wherein the packing unit stores the priority information and the encoded audio signal in the bit stream.
(20)
Generates priority information for audio signals of multiple channels or multiple objects at a given time,
A coding method that includes a step of storing the priority information in a bitstream.
(21)
Generates priority information for audio signals of multiple channels or multiple objects at a given time,
A program that causes a computer to perform a process including a step of storing the priority information in a bit stream.

11 Encoding device, 21 channel audio coding section, 22 object audio coding section, 23 metadata input section, 24 packing section, 51 coding section, 52 priority information generation section, 61 MDCT section, 91 coding section, 92 Priority information generation unit, 101 MDCT unit, 151 decoding device, 161 unpacking / decoding unit, 162 rendering unit, 163 mixing unit, 191 priority information acquisition unit, 193 channel audio signal decoding unit, 194 output selection unit, 196 IMDCT section, 198 object audio signal decoding section, 199 output selection section, 201 IMDCT section, 231 priority information generation section, 232 priority information generation section, 271 overlap addition section, 272 gain adjustment section, 273 SBR processing section, 274 Overlap processing unit, 275 gain adjustment unit, 276 SBR processing unit

The decoding device of one aspect of the present technology acquires the priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each of the objects from the supplied bit stream at a predetermined time. A unit and an audio signal decoding unit that controls the decoding process of the encoded audio signal of the object based on the priority information.

A decoding method or program of one aspect of the present technology obtains priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each object from a supplied bit stream at a predetermined time. A step of controlling the decoding process of the encoded audio signal of the object based on the priority information is included.

In one aspect of the present technology, priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each object at a predetermined time is acquired from the supplied bit stream, and the priority is obtained. The decoding process of the encoded audio signal of the object is controlled based on the degree information.

Claims (3)

An acquisition unit that acquires priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each object at a predetermined time from the supplied bit stream.
Based on the priority information, the encoded audio signal of the object whose priority indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority indicated in the priority information is the predetermined degree. A decoding device including an audio signal decoding unit that does not decode the encoded audio signal of the object that is less than the degree. The decryption device
The priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each object at a predetermined time is acquired from the supplied bit stream.
Based on the priority information, the encoded audio signal of the object whose priority indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority indicated in the priority information is the predetermined degree. A decoding method that does not decode the encoded audio signal of the object that is less than a degree. The priority information of the encoded audio signals of a plurality of objects and the encoded audio signals of each object at a predetermined time is acquired from the supplied bit stream.
Based on the priority information, the encoded audio signal of the object whose priority indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority indicated in the priority information is the predetermined degree. A program that causes a computer to perform a process that includes a step that does not decode the encoded audio signal of the object that is less than a degree.

Images