JP2019049745A - Decoder and method, and program - Google Patents

Abstract

To enable the amount of calculation for decoding an audio signal to be reduced.SOLUTION: A priority information acquisition unit acquires priority information of each channel from a bit stream for supply to an output selection unit. A channel audio signal decoding unit decodes encoded data of an audio signal of each channel, and supplies an obtained MDCT coefficient to the output selection unit. The output selection unit supplies the MDCT coefficient to an IMDCT unit in the case where the priority degree indicated in the priority information is more than or equal to a predetermined degree, and supplies 0 as the MDCT coefficient to a 0-value output unit in the case where the priority degree is less than the predetermined degree. The 0-value output unit generates an audio signal on the basis of 0 supplied as the MDCT coefficient. In addition, the IMDCT unit performs IMDCT on the basis of the MDCT coefficient to generate an audio signal. The present technology can be applied to an encoder and a decoder.SELECTED DRAWING: Figure 10

Description

  The present technology relates to a decoding device and method, and a program, and more particularly to a decoding device and method, and a program capable of reducing the amount of calculation of decoding of an audio signal.

  For example, as a method of encoding an audio signal, international standard standards such as MPEG (Moving Picture Experts Group) -2 AAC (Advanced Audio Coding), MPEG-4 AAC, MPEG-D USAC (Unified Speech and Audio Coding) Standard multi-channel coding is known (see, for example, Non-Patent Document 1 and Non-Patent Document 2).

INTERNATIONAL STANDARD ISO / IEC 14496-3 Fourth edition 2009-09-01 Information technology-coding of audio-visual objects-part3: Audio International STANDARD ISO / IEC 23003-3 Frist edition 2012-04-01 Information technology-coding of audio-visual objects-part 3: Unified speech and audio coding

  By the way, in order to achieve more realistic reproduction than conventional 5.1 channel surround reproduction and to transmit a plurality of sound materials (objects), a coding technology using more audio channels is required.

  For example, consider a case where a 24-channel audio signal and an audio signal of a plurality of objects are encoded and decoded, and a case where a 2-channel audio signal is encoded and decoded. In such a case, although it is possible to decode 2-channel audio signals in real time, such as in a mobile device with low computing power, it is possible to decode 24-channel audio signals and audio signals of multiple objects in real time. It may be difficult.

  In the current audio codec such as MPEG-D USAC, it is difficult to reduce the amount of calculation at the time of decoding since it is necessary to decode the audio signals of all channels and all objects. Then, depending on the device on the decoding side, it may not be possible to reproduce the audio signal in real time.

  The present technology has been made in view of such a situation, and is intended to be able to reduce the computational complexity of decoding.

  The decoding device according to one aspect of the present technology acquires encoded audio signals of a plurality of objects from the supplied bitstream and priority information of the encoded audio signals of each of the objects at a predetermined time. The encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree based on the priority information and the priority information indicated in the priority information And an audio signal decoding unit that does not decode the encoded audio signal of the object that is less than the predetermined degree.

  The decoding method or program according to one aspect of the present technology obtains encoded audio signals of a plurality of objects from a supplied bit stream and priority information of the encoded audio signals of each of the objects at a predetermined time. And decoding the encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree based on the priority information, and the priority degree indicated in the priority information is the The step of not decoding the encoded audio signal of the object which is less than a predetermined degree.

  In one aspect of the present technology, encoded audio signals of a plurality of objects and priority information at predetermined times of encoded audio signals of each of the objects are obtained from a supplied bit stream, and the priority information is obtained Based on the degree information, the encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority degree indicated in the priority information is less than the predetermined degree. The encoded audio signal of the object is not decoded.

It is a figure explaining a bit stream. It is a figure explaining encoding. It is a figure explaining priority information. It is a figure explaining the meaning of the value of priority information. It is a figure which shows the structural example of an encoding apparatus. It is a figure which shows the structural example of a channel audio encoding part. It is a figure which shows the structural example of an object audio encoding part. It is a flowchart explaining an encoding process. It is a figure which shows the structural example of a decoding apparatus. It is a figure which shows the structural example of a unpacking / decoding part. It is a flowchart explaining a decoding process. It is a flowchart explaining a selective decoding process. It is a figure which shows the other structural example of an unpacking / decoding part. It is a flowchart explaining a selective decoding process. It is a figure which shows an example of the syntax of the metadata of an object. It is a figure explaining generation of an audio signal. It is a figure explaining generation of an audio signal. It is a figure explaining selection of the output place of a MDCT coefficient. It is a figure explaining gain adjustment of an audio signal and a power value of a high region. It is a figure explaining gain adjustment of an audio signal and a power value of a high region. It is a figure which shows the other structural example of an unpacking / decoding part. It is a flowchart explaining a selective decoding process. It is a figure explaining gain adjustment of an audio signal. It is a figure explaining gain adjustment of an audio signal. It is a figure which shows the other structural example of an unpacking / decoding part. It is a flowchart explaining a selective decoding process. It is a figure explaining a VBAP gain. It is a figure explaining a VBAP gain. It is a figure which shows the other structural example of an unpacking / decoding part. It is a flowchart explaining a decoding process. It is a flowchart explaining a selective decoding process. It is a figure showing an example of composition of a computer.

  Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

First Embodiment
<About the outline of this technology>
The present technology transmits the priority information of the audio signal of each channel and the priority information of the audio signal of each object in the encoding of the audio signal of each channel constituting the multi-channel and the audio signal of the object. It is intended to be able to reduce the computational complexity of decoding.

  Further, the present technology performs frequency-time conversion on the decoding side when the priority indicated in the priority information of each channel or each object is equal to or higher than a predetermined degree, and the priority indicated in the priority information is a predetermined degree. In the case of less than this, it is possible to reduce the calculation amount of decoding of the audio signal by setting the result of the frequency time conversion to 0 without performing the frequency time conversion.

  In the following, the case where multi-channel audio signals and object audio signals are encoded according to the AAC standard will be described, but the same processing is performed when encoding is performed using another method.

  For example, when multi-channel audio signals and audio signals of a plurality of objects are encoded and transmitted according to the AAC standard, the audio signals of each channel and each object are encoded and transmitted frame by frame.

  Specifically, as shown in FIG. 1, encoded audio signals and information necessary for decoding of audio signals are stored in a plurality of elements (bit stream elements), and a bit stream composed of those elements is transmitted It will be done.

  In this example, in the bit stream of one frame, t elements EL1 to ELt are arranged in order from the head, and finally, an identifier TERM indicating the end position of the information of the frame is arranged. .

  For example, the element EL1 placed at the top is an ancillary data area called DSE (Data Stream Element), and the DSE describes information on each of a plurality of channels, such as information on downmixing of audio signals and identification information. Ru.

  The encoded audio signal is stored in the element EL2 to the element ELt following the element EL1.

  In particular, an element in which a single channel audio signal is stored is called SCE, and an element in which a pair of audio signals of paired channels is stored is called CPE. Also, the audio signal of each object is stored in SCE.

  In the present technology, the priority information of the audio signal of each channel constituting the multi-channel and the priority information of the audio signal of each object are generated and stored in the DSE.

  For example, as shown in FIG. 2, it is assumed that audio signals of consecutive frames F11 to F13 are encoded.

  In such a case, the encoding device (encoder) analyzes, for each of those frames, what degree of priority the audio signal of each channel is, for example, as shown in FIG. 3, the priority information of each channel Generate Similarly, the encoding device also generates priority information for the audio signal of each object.

  For example, the encoding apparatus analyzes how much priority the audio signal has, based on the sound pressure of the audio signal, the shape of the spectrum, and the correlation of the spectral shapes between the channels and objects.

  In FIG. 3, the priority information of each channel in the case where the total number of channels is M is shown as an example. That is, for each channel from the channel whose channel number is 0 to the channel whose channel number is M-1, a numerical value indicating the priority of the audio signal of those channels is shown as priority information.

  For example, the priority information of the channel whose channel number is 0 is 3 and the priority information of the channel whose channel number is 1 is 0. Hereinafter, a channel of a predetermined channel number m (m = 0, 1,..., M−1) is also referred to as a channel m.

  The value of the priority information shown in FIG. 3 is made to be any value from 0 to 7 as shown in FIG. 4, and when the value of the priority information is larger, at the time of reproducing the audio signal Priority, that is, the importance is high.

  Therefore, an audio signal having a priority information value of 0 has the lowest priority, and an audio signal having a priority information value of 7 has the highest priority.

  When a multi-channel audio signal or audio signals of a plurality of objects are simultaneously reproduced, the audio reproduced by the audio signals usually includes audio which is not as important as other audio. In other words, even if a specific sound is not reproduced in the entire sound, there is a sound that does not give the listener a sense of discomfort.

  Therefore, if the audio signal with low priority is not decoded as needed, it is possible to reduce the calculation amount of decoding while suppressing the deterioration of the sound quality. Therefore, in the encoding apparatus, the priority information indicating the degree of importance of each audio signal at the time of reproduction, that is, the degree to which decoding should be prioritized, can be set for each frame so that the audio signal not to be decoded can be appropriately selected. To each audio signal.

  As described above, when the priority information of each audio signal is determined, the priority information is stored in the DSE of the element EL1 shown in FIG. In particular, in the example of FIG. 3, since the number of channels constituting the multi-channel audio signal is M, the priority information of M channels of channel 0 to channel M-1 is stored in DSE.

  Similarly, priority information of each object is also stored in the DSE of the element EL1. Here, for example, assuming that there are N objects with object numbers of 0 to N-1, priority information is determined for each of the N objects, and stored in the DSE.

  Hereinafter, an object of a predetermined object number n (n = 0, 1,..., N−1) will be also referred to as an object n.

  As described above, if priority information is determined for each audio signal, which audio signal is important at the time of reproduction on the reproduction side, that is, the decoding side of the audio signal, and is preferentially decoded, that is, used for reproduction. You can easily identify what to do.

  Returning to the explanation of FIG. 2, for example, it is assumed that the priority information of the audio signal of the frame F11 and the frame F13 of the predetermined channel is 7, and the priority information of the audio signal of the frame F12 of the predetermined channel is 0.

  Further, it is assumed that decoding is not performed on the audio signal decoding side, that is, an audio signal less than a predetermined degree of priority on the decoding device (decoder).

  Here, for example, assuming that the predetermined priority is referred to as a threshold, and the threshold is 4, in the above-described example, audio signals of frames F11 and F13 of a predetermined channel whose priority information is 7 are used. Is decrypted.

  On the other hand, decoding is not performed on the audio signal of the frame F12 of the predetermined channel in which the priority information is 0.

  Therefore, in this example, the audio signal of the frame F12 is a silence signal, and the audio signals of the frame F11 and the frame F13 are synthesized to be an audio signal of a final predetermined channel.

  More specifically, for example, at the time of encoding each audio signal, time-frequency conversion is performed on the audio signal, information obtained by the time-frequency conversion is encoded, and the resulting encoded data is stored in the element Ru.

  Although any processing may be performed as time frequency conversion, the description will be continued on the assumption that MDCT (Modified Discrete Cosine Transform) (Modified Discrete Cosine Transform) is performed as time frequency conversion.

  Further, in the decoding apparatus, decoding is performed on the encoded data, and an IMDC (Inverse Modified Discrete Cosine Transform) (inverse modified discrete cosine transformation) is performed on the MDCT coefficients obtained as a result, thereby generating an audio signal. . That is, here, IMDCT is performed as inverse conversion (frequency-time conversion) of time-frequency conversion.

  Therefore, in more detail, IMDCT is performed on the frame F11 and the frame F13 in which the priority information is the threshold value 4 or more, and an audio signal is generated.

  Further, the IMDCT is not performed for the frame F12 in which the priority information is less than the threshold value 4, and the result of the IMDCT is set to 0 to generate an audio signal. As a result, the audio signal of the frame F12 becomes a silence signal, that is, 0 data.

  As yet another example, in the example shown in FIG. 3, when the threshold is 4, among the audio signals of channels 0 to M-1, the channel whose priority information has a value less than 4 which is the threshold is used. Decoding of the 0, channel 1 and channel M-2 audio signals will not be performed.

  As described above, according to the comparison result with the threshold value, the audio signal with a low degree of priority indicated by the priority information is not decoded, thereby minimizing the deterioration of the sound quality while calculating the decoding. The amount can be reduced.

<Configuration Example of Encoding Device>
Next, specific embodiments of the encoding device and the decoding device to which the present technology is applied will be described. First, the coding apparatus will be described.

  FIG. 5 is a diagram illustrating a configuration example of a coding device to which the present technology is applied.

  The encoding apparatus 11 of FIG. 5 includes a channel audio encoding unit 21, an object audio encoding unit 22, a metadata input unit 23, and a packing unit 24.

  The channel audio encoding unit 21 is supplied with an audio signal of each channel of multi-channels in which the number of channels is M. For example, audio signals of each channel are supplied from microphones corresponding to those channels. In FIG. 5, characters “# 0” to “# M−1” represent channel numbers of the respective channels.

  The channel audio encoding unit 21 encodes the supplied audio signal of each channel, generates priority information based on the audio signal, and generates encoded data obtained by the encoding and the priority information. Supply to the packing unit 24.

  Audio signals of N objects are supplied to the object audio encoding unit 22. For example, the audio signal of each object is supplied from a microphone attached to the object. In FIG. 5, characters “# 0” to “# N−1” represent object numbers of the respective objects.

  The object audio encoding unit 22 encodes the audio signal of each object supplied, generates priority information based on the audio signal, and generates encoded data obtained by the encoding and the priority information. Supply to the packing unit 24.

  The metadata input unit 23 supplies metadata of each object to the packing unit 24. For example, metadata of an object is spatial position information indicating the position of the object in space. More specifically, for example, the spatial position information is three-dimensional coordinate information indicating the coordinates of the position of the object in the three-dimensional space.

  The packing unit 24 is supplied with the encoded data and priority information supplied from the channel audio encoding unit 21, the encoded data and priority information supplied from the object audio encoding unit 22, and the metadata input unit 23. Pack the metadata to generate and output a bitstream.

  The bit stream thus obtained includes, for each frame, coded data of each channel, priority information of each channel, coded data of each object, priority information of each object, and metadata of each object. It will be done.

  Here, the audio signals of M channels and the audio signals of N objects stored in the bit stream of one frame are audio signals of the same frame to be reproduced simultaneously.

  Here, an example in which priority information is generated for each audio signal for each frame as priority information of audio signals of each channel and each object will be described, but an arbitrary predetermined time is used as a unit. For example, one priority information may be generated for an audio signal of several frames.

<Configuration Example of Channel Audio Encoding Unit>
Also, the channel audio encoding unit 21 of FIG. 5 is configured in more detail, for example, as shown in FIG.

  The channel audio coding unit 21 shown in FIG. 6 includes a coding unit 51 and a priority information generation unit 52.

  The encoding unit 51 includes an MDCT unit 61. The encoding unit 51 encodes an audio signal of each channel supplied from the outside.

  That is, the MDCT unit 61 performs MDCT on the audio signal of each channel supplied from the outside. The encoding unit 51 encodes the MDCT coefficients of each channel obtained by the MDCT, and supplies the encoded data of each channel obtained as a result, that is, the encoded audio signal to the packing unit 24.

  In addition, the priority information generation unit 52 analyzes the audio signal of each channel supplied from the outside, generates priority information of the audio signal of each of those channels, and supplies the priority information to the packing unit 24.

<Configuration Example of Object Audio Encoding Unit>
Furthermore, the object audio encoding unit 22 of FIG. 5 is configured in more detail, for example, as shown in FIG.

  The object audio encoding unit 22 shown in FIG. 7 includes an encoding unit 91 and a priority information generation unit 92.

  The encoding unit 91 includes an MDCT unit 101. The encoding unit 91 encodes an audio signal of each object supplied from the outside.

  That is, the MDCT unit 101 performs MDCT on the audio signal of each object supplied from the outside. The encoding unit 91 encodes the MDCT coefficients of each object obtained by the MDCT, and supplies the encoded data of each object obtained as a result, that is, the encoded audio signal to the packing unit 24.

  Also, the priority information generation unit 92 analyzes the audio signal of each object supplied from the outside, generates priority information of the audio signal of each object, and supplies the priority information to the packing unit 24.

<Description of encoding process>
Next, the process performed by the encoding device 11 will be described.

  The encoding device 11 performs encoding processing when audio signals of a plurality of channels and audio signals of a plurality of objects, which are simultaneously reproduced, are supplied for one frame, the encoded audio signal is Output the included bitstream.

  Hereinafter, the encoding process by the encoding device 11 will be described with reference to the flowchart in FIG. This encoding process is performed for each frame of the audio signal.

  In step S 11, the priority information generation unit 52 of the channel audio encoding unit 21 generates priority information of the supplied audio signal of each channel and supplies the generated priority information to the packing unit 24. For example, the priority information generation unit 52 analyzes the audio signal for each channel, and generates priority information based on the sound pressure and the spectral shape of the audio signal, the correlation of the spectral shapes between the channels, and the like.

  In step S12, the packing unit 24 stores the priority information of the audio signal of each channel supplied from the priority information generation unit 52 in the DSE of the bit stream. That is, priority information is stored in the first element of the bitstream.

  In step S 13, the priority information generation unit 92 of the object audio encoding unit 22 generates priority information of the supplied audio signal of each object, and supplies the priority information to the packing unit 24. For example, the priority information generation unit 92 analyzes the audio signal for each object, and generates priority information based on the sound pressure and the spectral shape of the audio signal, the correlation of the spectral shape between the objects, and the like.

  When the priority information of the audio signal of each channel or each object is generated, the number of audio signals to which the degree of priority is assigned corresponds to the number of channels or the number of objects for each priority which is the value of the priority information. It may be determined in advance.

  For example, in the example of FIG. 3, the number of audio signals for which the priority information is "7", that is, the number of channels is five, the number of audio signals for which the priority information is "6" is three, It may be set in advance.

  In step S14, the packing unit 24 stores the priority information of the audio signal of each object supplied from the priority information generation unit 92 in the DSE of the bit stream.

  In step S15, the packing unit 24 stores the metadata of each object in the DSE of the bit stream.

  For example, the metadata input unit 23 acquires metadata of each object by receiving an input operation of the user, performing communication with the outside, or reading from an external recording area, and the packing unit 24 Supply to The packing unit 24 stores the metadata thus supplied from the metadata input unit 23 in the DSE.

  As a result of the above processing, priority information of audio signals of all channels, priority information of audio signals of all objects, and metadata of all objects are stored in the bit stream DSE.

  In step S16, the encoding unit 51 of the channel audio encoding unit 21 encodes the supplied audio signal of each channel.

  More specifically, the MDCT unit 61 performs MDCT on the audio signal of each channel, and the encoding unit 51 encodes the MDCT coefficient of each channel obtained by MDCT, and the result for each channel The encoded data is supplied to the packing unit 24.

  In step S17, the packing unit 24 stores the encoded data of the audio signal of each channel supplied from the encoding unit 51 in the SCE or CPE of the bit stream. That is, encoded data is stored in each element arranged subsequent to DSE in the bitstream.

  In step S18, the encoding unit 91 of the object audio encoding unit 22 encodes the supplied audio signal of each object.

  More specifically, the MDCT unit 101 performs an MDCT on the audio signal of each object, and the encoding unit 91 encodes the MDCT coefficients of each object obtained by the MDCT, and the result of each object The encoded data is supplied to the packing unit 24.

  In step S19, the packing unit 24 stores the encoded data of the audio signal of each object supplied from the encoding unit 91 in the SCE of the bit stream. That is, encoded data is stored in some elements arranged after DSE in the bitstream.

  By the above processing, the priority information and encoded data of audio signals of all channels, the priority information and encoded data of audio signals of all objects, and the metadata of all objects are stored for the frame to be processed. The obtained bitstream is obtained.

  In step S20, the packing unit 24 outputs the obtained bit stream, and the encoding process ends.

  As described above, the encoding device 11 generates the priority information of the audio signal of each channel and the priority information of the audio signal of each object, stores the information in the bit stream, and outputs the bit stream. Therefore, on the decoding side, it can be easily grasped which audio signal has a higher degree of priority.

  Thus, the decoding side can selectively decode the encoded audio signal according to the priority information. As a result, it is possible to reduce the amount of calculation for decoding while minimizing deterioration in the sound quality of the sound reproduced by the audio signal.

  In particular, by storing the priority information of the audio signal of each object in the bit stream, not only the calculation amount of decoding can be reduced but also the calculation amount of subsequent processing such as rendering can be reduced on the decoding side. it can.

<Configuration Example of Decoding Device>
Next, a decoding apparatus will be described which receives the bit stream output from the encoding apparatus 11 described above and decodes the encoded data contained in the bit stream.

  Such a decoding apparatus is configured, for example, as shown in FIG.

  The decoding device 151 illustrated in FIG. 9 includes an unpacking / decoding unit 161, a rendering unit 162, and a mixing unit 163.

  The unpacking / decoding unit 161 obtains the bit stream output from the encoding device 11, and performs unpacking and decoding of the bit stream.

  The unpacking / decoding unit 161 supplies the audio signal of each object obtained by the unpacking and decoding and the metadata of each object to the rendering unit 162. At this time, the unpacking / decoding unit 161 decodes the encoded data of each object according to the priority information included in the bit stream.

  Also, the unpacking / decoding unit 161 supplies the audio signal of each channel obtained by the unpacking and the decoding to the mixing unit 163. At this time, the unpacking / decoding unit 161 decodes the encoded data of each channel according to the priority information included in the bit stream.

  The rendering unit 162 generates an M-channel audio signal based on the audio signal of each object supplied from the unpacking / decoding unit 161 and the spatial position information as metadata of each object, and supplies this to the mixing unit 163. Do. At this time, the rendering unit 162 generates audio signals of M channels so that the sound image of each object is localized at the position indicated by the spatial position information of the objects.

  The mixing unit 163 performs weighted addition on the audio signal of each channel supplied from the unpacking / decoding unit 161 and the audio signal of each channel supplied from the rendering unit 162 for each channel to obtain each final channel. Generate an audio signal of The mixing unit 163 supplies the audio signal of each final channel obtained in this manner to a speaker corresponding to each external channel to reproduce sound.

<Configuration Example of Unpacking / Decoding Unit>
Further, the unpacking / decoding unit 161 of the decoding device 151 shown in FIG. 9 is configured in more detail, for example, as shown in FIG.

  The unpacking / decoding unit 161 shown in FIG. 10 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, an object audio. A signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a zero value output unit 200, and an IMDCT unit 201 are included.

  The priority information acquisition unit 191 acquires the priority information of the audio signal of each channel from the supplied bit stream and supplies it to the output selection unit 194, and the priority information of the audio signal of each object from the bit stream. It acquires and supplies to the output selection part 199.

  In addition, the priority information acquisition unit 191 acquires metadata of each object from the supplied bit stream and supplies the metadata to the rendering unit 162, and the bit stream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197. Supply.

  The channel audio signal acquisition unit 192 acquires encoded data of each channel from the bit stream supplied from the priority information acquisition unit 191, and supplies the encoded data to the channel audio signal decoding unit 193. The channel audio signal decoding unit 193 decodes the encoded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the MDCT coefficients obtained as a result to the output selection unit 194.

  The output selection unit 194 selectively switches the output destination of the MDCT coefficient of each channel supplied from the channel audio signal decoding unit 193 based on the priority information of each channel supplied from the priority information acquisition unit 191.

  That is, when the priority information for a predetermined channel is less than the predetermined threshold P, the output selection unit 194 supplies the MDCT coefficient of the channel as 0 to the 0 value output unit 195. Further, when the priority information for a predetermined channel is equal to or higher than the predetermined threshold P, the output selection unit 194 supplies the MDCT coefficients of the channel supplied from the channel audio signal decoding unit 193 to the IMDCT unit 196.

  The zero value output unit 195 generates an audio signal based on the MDCT coefficients supplied from the output selection unit 194, and supplies the audio signal to the mixing unit 163. In this case, since the MDCT coefficient is 0, a silent audio signal is generated.

  The IMDCT unit 196 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 194 to generate an audio signal, and supplies the audio signal to the mixing unit 163.

  The object audio signal acquisition unit 197 acquires encoded data of each object from the bit stream supplied from the priority information acquisition unit 191, and supplies the encoded data to the object audio signal decoding unit 198. The object audio signal decoding unit 198 decodes the encoded data of each object supplied from the object audio signal acquisition unit 197, and supplies the MDCT coefficients obtained as a result to the output selection unit 199.

  The output selection unit 199 selectively switches the output destination of the MDCT coefficient of each object supplied from the object audio signal decoding unit 198 based on the priority information of each object supplied from the priority information acquisition unit 191.

  That is, when the priority information on a predetermined object is less than the predetermined threshold value Q, the output selection unit 199 supplies the MDCT coefficient of the object as 0 to the 0 value output unit 200. Further, when the priority information of a predetermined object is equal to or higher than a predetermined threshold value Q, the output selection unit 199 supplies the MDCT coefficients of the object supplied from the object audio signal decoding unit 198 to the IMDCT unit 201.

  Note that the value of the threshold Q may be the same as the value of the threshold P, or may be different from the value of the threshold P. By appropriately setting the threshold P and the threshold Q according to the calculation capability of the decoding device 151, the amount of calculation of decoding of the audio signal is reduced to the amount of calculation within the range in which the decoding device 151 can decode in real time. It can be done.

  The zero-value output unit 200 generates an audio signal based on the MDCT coefficients supplied from the output selection unit 199 and supplies the audio signal to the rendering unit 162. In this case, since the MDCT coefficient is 0, a silent audio signal is generated.

  The IMDCT unit 201 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 199 to generate an audio signal, and supplies the audio signal to the rendering unit 162.

<Description of decryption processing>
Next, the operation of the decoding device 151 will be described.

  When the bit stream for one frame is supplied from the encoding device 11, the decoding device 151 performs a decoding process to generate an audio signal, and outputs the audio signal to the speaker. The decoding process performed by the decoding device 151 will be described below with reference to the flowchart in FIG.

  In step S51, the unpacking / decoding unit 161 acquires the bit stream transmitted from the encoding device 11. That is, a bitstream is received.

  In step S52, the unpacking / decoding unit 161 performs selective decoding processing.

  Although details of the selective decoding process will be described later, in the selective decoding process, coded data of each channel and coded data of each object are selectively decoded based on priority information. Then, the audio signal of each channel obtained as a result is supplied to the mixing unit 163, and the audio signal of each object is supplied to the rendering unit 162. Also, the metadata of each object acquired from the bit stream is supplied to the rendering unit 162.

  In step S53, the rendering unit 162 renders the audio signal of each object based on the audio signal of each object supplied from the unpacking / decoding unit 161 and the spatial position information as metadata of each object.

  For example, the rendering unit 162 generates an audio signal of each channel so that the sound image of each object is localized at the position indicated by the spatial position information by VBAP (Vector Base Amplitude Panning) based on the spatial position information. Supply to

  In step S 54, the mixing unit 163 performs weighted addition of the audio signal of each channel supplied from the unpacking / decoding unit 161 and the audio signal of each channel supplied from the rendering unit 162 for each channel, and the external speaker Supply to As a result, audio signals of channels corresponding to the speakers are supplied to the speakers, so that the speakers reproduce sound based on the supplied audio signals.

  When the audio signal of each channel is supplied to the speaker, the decoding process ends.

  As described above, the decoding device 151 acquires the priority information from the bit stream, and decodes the encoded data of each channel and each object according to the priority information.

<Description of selective decoding process>
Subsequently, selective decoding processing corresponding to the processing in step S52 in FIG. 11 will be described with reference to the flowchart in FIG.

  In step S81, the priority information acquisition unit 191 acquires, from the supplied bit stream, the priority information of the audio signal of each channel and the priority information of the audio signal of each object, and outputs the output selection unit 194 and the like. The signal is supplied to the output selection unit 199.

  Also, the priority information acquisition unit 191 acquires metadata of each object from the bit stream and supplies the metadata to the rendering unit 162, and supplies the bit stream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197.

  In step S82, the channel audio signal acquisition unit 192 sets 0 as the channel number of the channel to be processed and holds the channel number.

  In step S83, the channel audio signal acquisition unit 192 determines whether the channel number held is less than the number M of channels.

  When it is determined in step S83 that the channel number is less than M, in step S84, the channel audio signal decoding unit 193 decodes the encoded data of the audio signal of the channel to be processed.

  That is, the channel audio signal acquisition unit 192 acquires encoded data of the channel to be processed from the bit stream supplied from the priority information acquisition unit 191, and supplies the encoded data to the channel audio signal decoding unit 193.

  Then, the channel audio signal decoding unit 193 decodes the encoded data supplied from the channel audio signal acquisition unit 192, and supplies the MDCT coefficients obtained as a result to the output selection unit 194.

  In step S85, the output selection unit 194 determines whether the priority information of the processing target channel supplied from the priority information acquisition unit 191 is equal to or higher than the threshold P specified by the upper control device or the like (not shown). Determine Here, the threshold value P is determined, for example, according to the calculation capability of the decoding device 151 or the like.

  If it is determined in step S85 that the priority information is equal to or higher than the threshold P, the output selection unit 194 supplies the MDCT coefficients of the processing target channel supplied from the channel audio signal decoding unit 193 to the IMDCT unit 196. The process proceeds to step S86. In this case, since the priority of the audio signal of the channel to be processed is equal to or higher than the predetermined priority, decoding for the channel, more specifically IMDCT, is performed.

  In step S 86, the IMDCT unit 196 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 194 to generate an audio signal of the channel to be processed, and supplies the audio signal to the mixing unit 163. After the audio signal is generated, the process proceeds to step S87.

  On the other hand, when it is determined in step S85 that the priority information is less than the threshold value P, the output selection unit 194 supplies the MDCT coefficient as 0 to the 0 value output unit 195.

  The zero-value output unit 195 generates an audio signal of the channel to be processed from the MDCT coefficient that is 0 supplied from the output selection unit 194, and supplies the audio signal to the mixing unit 163. Therefore, in the zero-value output unit 195, substantially no processing for generating an audio signal such as IMDCT is performed.

  The audio signal generated by the zero value output unit 195 is a silence signal. After the audio signal is generated, the process proceeds to step S87.

  If it is determined in step S85 that the priority information is less than the threshold value P, or an audio signal is generated in step S86, the channel audio signal acquisition unit 192 holds 1 for the channel number held in step S87. In addition, the channel number of the channel to be processed is updated.

  After the channel number is updated, the process returns to step S83, and the above-described process is repeated. That is, an audio signal of a new processing target channel is generated.

  If it is determined in step S83 that the channel number of the processing target channel is not less than M, audio signals are obtained for all the channels, so the process proceeds to step S88.

  In step S88, the object audio signal acquisition unit 197 sets 0 as the object number of the object to be processed and holds the object number.

  In step S89, the object audio signal acquisition unit 197 determines whether the stored object number is less than the number N of objects.

  If it is determined in step S89 that the object number is less than N, in step S90, the object audio signal decoding unit 198 decodes the encoded data of the audio signal of the object to be processed.

  That is, the object audio signal acquisition unit 197 acquires encoded data of the object to be processed from the bit stream supplied from the priority information acquisition unit 191, and supplies the encoded data to the object audio signal decoding unit 198.

  Then, the object audio signal decoding unit 198 decodes the encoded data supplied from the object audio signal acquisition unit 197, and supplies the MDCT coefficients obtained as a result to the output selection unit 199.

  In step S 91, the output selection unit 199 determines whether the priority information of the object to be processed supplied from the priority information acquisition unit 191 is equal to or higher than the threshold Q designated by the upper control device or the like (not shown). Determine Here, the threshold value Q is determined, for example, according to the calculation capability of the decoding device 151 or the like.

  In step S 91, when it is determined that the priority information is equal to or higher than the threshold Q, the output selection unit 199 supplies the MDCT coefficient of the object to be processed supplied from the object audio signal decoding unit 198 to the IMDCT unit 201. The process proceeds to step S92.

  In step S 92, the IMDCT unit 201 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 199 to generate an audio signal of the object to be processed, and supplies the audio signal to the rendering unit 162. After the audio signal is generated, the process proceeds to step S93.

  On the other hand, when it is determined in step S91 that the priority information is less than the threshold value Q, the output selection unit 199 supplies the MDCT coefficient as 0 to the 0 value output unit 200.

  The zero-value output unit 200 generates an audio signal of the object to be processed from the MDCT coefficient that is 0 supplied from the output selection unit 199, and supplies the audio signal to the rendering unit 162. Therefore, in the zero-value output unit 200, substantially no processing for generating an audio signal such as IMDCT is performed.

  The audio signal generated by the zero value output unit 200 is a silence signal. After the audio signal is generated, the process proceeds to step S93.

  If it is determined in step S91 that the priority information is less than the threshold Q, or an audio signal is generated in step S92, the object audio signal acquisition unit 197 sets the object number held by 1 to 1 in step S93. In addition, the object number of the object to be processed is updated.

  After the object number is updated, the process returns to step S89, and the above-described process is repeated. That is, an audio signal of a new object to be processed is generated.

  If it is determined in step S89 that the object number of the object to be processed is not less than N, audio signals are obtained for all the channels and objects, so the selective decoding process ends, and then the process shown in FIG. The process proceeds to step S53.

  As described above, does the decoding device 151 compare the priority information and the threshold for each channel or each object, and decode the audio signal encoded for each channel or object of the frame to be processed? The encoded audio signal is decoded while determining whether or not it is not.

  That is, in the decoding device 151, the encoded audio signal is decoded by a predetermined number corresponding to the priority information of each audio signal, and the remaining audio signal is not decoded.

  As a result, only the audio signal having a high degree of priority can be selectively decoded according to the reproduction environment, and the calculation amount of decoding can be reduced while minimizing the deterioration of the sound quality of the sound reproduced by the audio signal. be able to.

  Moreover, by decoding the encoded audio signal based on the priority information of the audio signal of each object, not only the calculation amount of decoding of the audio signal but also the subsequent processing such as the processing in the rendering unit 162 etc. The computational complexity of can also be reduced.

<Modified Example 1 of First Embodiment>
<About priority information>
Although it has been described above that one priority information is generated for one audio signal of each channel or each object, a plurality of priority information may be generated.

  In such a case, for example, a plurality of pieces of priority information are generated for each of the computing capabilities according to the computational complexity of the decoding, that is, the computing capabilities of the decoding side.

  Specifically, for example, on the basis of the amount of calculation for decoding audio signals corresponding to two channels in real time, priority information for a device having a calculation capability equivalent to two channels is generated.

  In the priority information for such a two-channel equivalent device, for example, among all the audio signals, the priority is low so that the number of audio signals to which a value having a lower priority, that is, close to 0, is allocated as priority information increases. Information is generated.

  Also, for example, based on the amount of calculation for decoding an audio signal equivalent to 24 channels in real time, priority information for an apparatus having a calculation capability equivalent to 24 channels is also generated. In the priority information for a device corresponding to 24 channels, for example, priority information is generated such that among all audio signals, a higher priority, that is, a value closer to 7 increases the number of audio signals allocated as priority information. Be done.

  In this case, for example, in step S11 of FIG. 8, the priority information generating unit 52 generates priority information for a device corresponding to two channels with respect to the audio signal of each channel, and An identifier indicating that the device corresponds to a channel is added and supplied to the packing unit 24.

  Furthermore, in step S11, the priority information generation unit 52 also generates priority information for the device corresponding to 24 channels for the audio signal of each channel, and in the priority information of the device corresponding to 24 channels, An identifier indicating that it is for the purpose is added and supplied to the packing unit 24.

  Similarly, in step S13 of FIG. 8, the priority information generation unit 92 generates priority information for devices corresponding to two channels and priority information for devices corresponding to 24 channels, and adds an identifier. , Supply to the packing unit 24.

  As a result, for example, a plurality of pieces of priority information can be obtained according to the calculation capability of a playback device such as a portable audio player, a multi-function mobile phone, a tablet computer, a television receiver, a personal computer, or high-quality audio device. become.

  For example, since playback devices such as portable audio players have relatively low computing power, such playback devices may decode the encoded audio signal based on the priority information for the two-channel equivalent device, The audio signal can be played back in real time.

  As described above, when a plurality of pieces of priority information are generated for one audio signal, the decoding device 151 uses, for example, any priority information of the plurality of pieces of priority information by the higher-level control device. It is instructed to the priority information acquisition unit 191 or the like whether to perform decryption. The indication of which priority information to use is performed, for example, by supplying an identifier.

  Note that which identifier priority information to use may be determined in advance for each decoding device 151.

  In example priority information acquisition unit 191, when either using priority information in advance which identifiers have been established, or if the identifier is specified by the upper controller, in step S81 in FIG. 12, the priority information acquisition unit A step 191 obtains priority information to which a predetermined identifier is added. Then, the acquired priority information is supplied from the priority information acquisition unit 191 to the output selection unit 194 and the output selection unit 199.

  In other words, among the plurality of pieces of priority information stored in the bit stream, one suitable piece of priority information is selected in accordance with the calculation capability and the like of the decoding device 151, more specifically, the unpacking / decoding unit 161. Be done.

  In this case, different identifiers may be used for the priority information of each channel and the priority information of each object, and the priority information may be read from the bitstream.

  As described above, by selecting and acquiring specific priority information from among a plurality of pieces of priority information included in the bit stream, appropriate priority information is selected according to the calculation capability of the decoding device 151 and the like. It can be selected and decoded. As a result, the audio signal can be reproduced in real time in any of the decoding devices 151.

Second Embodiment
<Configuration Example of Unpacking / Decoding Unit>
In the above, an example in which priority information is included in the bit stream output from the encoding device 11 has been described, but depending on the encoding device, priority information may not be included in the bit stream. possible.

  Therefore, the priority information may be generated in the decoding device 151. For example, priority information can be generated using information indicating the sound pressure of the audio signal and information indicating the spectral shape, which can be extracted from encoded data of the audio signal included in the bit stream.

  As described above, when the priority information is generated in the decoding device 151, the unpacking / decoding unit 161 of the decoding device 151 is configured, for example, as illustrated in FIG. In FIG. 13, parts corresponding to the case in FIG. 10 are given the same reference numerals, and the description thereof will be omitted as appropriate.

  The unpacking / decoding unit 161 shown in FIG. 13 includes a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, an object audio signal acquisition unit 197, and an object audio. A signal decoding unit 198, an output selection unit 199, a zero value output unit 200, an IMDCT unit 201, a priority information generation unit 231, and a priority information generation unit 232 are included.

  The configuration of the unpacking / decoding unit 161 shown in FIG. 13 is that the priority information acquisition unit 191 is not provided, and a priority information generation unit 231 and a priority information generation unit 232 are newly provided. Unlike the unpacking / decoding unit 161 of FIG. 10, the other configuration is the same as the unpacking / decoding unit 161 of FIG.

  The channel audio signal acquisition unit 192 acquires encoded data of each channel from the supplied bit stream, and supplies the encoded data to the channel audio signal decoding unit 193 and the priority information generation unit 231.

  The priority information generation unit 231 generates priority information of each channel based on the encoded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the priority information to the output selection unit 194.

  The object audio signal acquisition unit 197 acquires encoded data of each object from the supplied bit stream, and supplies the encoded data to the object audio signal decoding unit 198 and the priority information generation unit 232. Also, the object audio signal acquisition unit 197 acquires metadata of each object from the supplied bit stream, and supplies the metadata to the rendering unit 162.

  The priority information generation unit 232 generates priority information of each object based on the encoded data of each object supplied from the object audio signal acquisition unit 197, and supplies the generated priority information to the output selection unit 199.

<Description of selective decoding process>
When the unpacking / decoding unit 161 has the configuration shown in FIG. 13, the decoding device 151 performs the selective decoding process shown in FIG. 14 as a process corresponding to step S52 of the decoding process shown in FIG. The selective decoding process by the decoding device 151 will be described below with reference to the flowchart in FIG.

  In step S131, the priority information generation unit 231 generates priority information of the audio signal of each channel.

  For example, the channel audio signal acquisition unit 192 acquires encoded data of each channel from the supplied bit stream, and supplies the encoded data to the channel audio signal decoding unit 193 and the priority information generation unit 231.

  The priority information generation unit 231 generates priority information of each channel based on the encoded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the priority information to the output selection unit 194.

  For example, the bit stream includes, as encoded data of an audio signal, a scale factor for obtaining MDCT coefficients, side information, and a quantization spectrum. Here, the scale factor is information indicating the sound pressure of the audio signal, and the quantization spectrum is information indicating the spectral shape of the audio signal.

  The priority information generation unit 231 generates priority information of the audio signal of each channel based on the scale factor and the quantization spectrum included as the encoded data of each channel. As described above, if priority information is generated using a scale factor or a quantization spectrum, priority information can be obtained immediately before decoding encoded data, and calculation for generating priority information is performed. The amount can also be reduced.

  The priority information may be generated based on the sound pressure of the audio signal and the spectral shape of the audio signal obtained from the peak envelope of the MDCT coefficients, which are also obtained by calculating the root mean square value of the MDCT coefficients. You may In this case, the priority information generation unit 231 appropriately decodes encoded data, and acquires MDCT coefficients from the channel audio signal decoding unit 193.

  After the priority information of each channel is obtained, the processes of steps S132 to S137 are performed, but these processes are the same as the processes of steps S82 to S87 of FIG. . However, in this case, since the encoded data of each channel is already obtained, only decoding of the encoded data is performed in step S134.

  If it is determined in step S133 that the channel number is not less than M, in step S138, the priority information generation unit 232 generates priority information of the audio signal of each object.

  For example, the object audio signal acquisition unit 197 acquires encoded data of each object from the supplied bit stream, and supplies the encoded data to the object audio signal decoding unit 198 and the priority information generation unit 232. Also, the object audio signal acquisition unit 197 acquires metadata of each object from the supplied bit stream, and supplies the metadata to the rendering unit 162.

  The priority information generation unit 232 generates priority information of each object based on the encoded data of each object supplied from the object audio signal acquisition unit 197, and supplies the priority information to the output selection unit 199. For example, the priority information is generated based on the scale factor and the quantization spectrum as in the case of each channel.

  In addition, priority information may be generated based on sound pressure or spectrum shape obtained from MDCT coefficients. In this case, the priority information generation unit 232 appropriately decodes the encoded data, and acquires MDCT coefficients from the object audio signal decoding unit 198.

  After the priority information of each object is obtained, the processes of steps S139 to S144 are performed and the selective decoding process ends, but these processes are similar to the processes of steps S88 to S93 of FIG. Therefore, the explanation is omitted. However, in this case, since the encoded data of each object is already acquired, only decoding of the encoded data is performed in step S141.

  After the selective decoding process is completed, the process proceeds to step S53 in FIG.

  As described above, the decoding device 151 generates the priority information of the audio signal of each channel or each object based on the encoded data included in the bit stream. By thus generating the priority information in the decoding device 151, it is possible to obtain appropriate priority information for each audio signal with a small amount of calculation, and to reduce the amount of calculation of decoding and the amount of calculation such as rendering. it can. It is also possible to minimize the deterioration of the sound quality of the audio reproduced by the audio signal.

  Although the priority information acquisition unit 191 of the unpacking / decoding unit 161 shown in FIG. 10 tries to acquire the priority information of the audio signal of each channel and each object from the supplied bit stream, the bit stream If priority information can not be acquired from the above, priority information may be generated. In such a case, the priority information acquisition unit 191 performs the same process as the priority information generation unit 231 and the priority information generation unit 232, and generates priority information of audio signals of each channel and each object from encoded data. Generate

Third Embodiment
<Threshold of priority information>
Furthermore, in the above, it has been described that, for each channel or each object, an audio signal to be decoded by comparing the priority information with the threshold value P or the threshold value Q, more specifically, MDCT coefficients for performing IMDCT are selected. The threshold P and the threshold Q may be dynamically changed for each frame of the audio signal.

  For example, the priority information acquisition unit 191 of the unpacking / decoding unit 161 illustrated in FIG. 10 can acquire the priority information of each channel and each object from the bit stream without requiring decoding.

  Therefore, for example, if the priority information acquisition unit 191 reads out the priority information of the audio signals of all the channels, it is possible to obtain the distribution of the priority information in the frame to be processed. In addition, the decoding apparatus 151 knows in advance its own calculation capability, such as how many channels it can process simultaneously, that is, in real time.

  Therefore, based on the distribution of priority information in the frame to be processed and the calculation capability of the decoding device 151, the priority information acquisition unit 191 determines the threshold P of the priority information on the frame to be processed. It is also good.

  For example, the threshold P is determined such that the largest number of audio signals can be decoded within the range in which the decoding device 151 can perform processing in real time.

  In addition, the priority information acquisition unit 191 can dynamically determine the threshold Q as in the case of the threshold P. In this case, the priority information acquisition unit 191 obtains the distribution of the priority information based on the priority information of the audio signals of all the objects, and processes based on the obtained distribution and the calculation capability of the decoding device 151 A threshold Q of priority information for a target frame is determined.

  Such determination of the threshold P and the threshold Q can be performed with a relatively small amount of calculation.

  By dynamically changing the threshold of the priority information in this manner, it is possible to minimize the deterioration of the sound quality of the sound reproduced by the audio signal while performing the decoding in real time. In such a case, in particular, it is not necessary to prepare a plurality of pieces of priority information, and it is not necessary to provide an identifier in the priority information, so that the code amount of the bit stream can be reduced.

<About object metadata>
Furthermore, in the first to third embodiments described above, it has been described that the first element of the bit stream stores metadata, priority information, and the like of an object for one frame. .

  In this case, the syntax of the portion where the object metadata and priority information are stored at the top element of the bit stream is, for example, as shown in FIG.

  In the example shown in FIG. 15, the spatial position information and the priority information of the object are stored for one frame in the metadata of the object.

  In this example, "num_objects" indicates the number of objects. Also, “object_priority [o]” indicates the priority information of the O-th object. Here, the O-th object is an object specified by an object number.

  “Position_azimuth [o]” indicates a horizontal angle representing the three-dimensional spatial position of the O-th object viewed from the user who is the viewer, that is, viewed from the predetermined reference position. Further, “position_elevation [o]” indicates a vertical direction angle representing the three-dimensional spatial position of the O-th object viewed from the user who is the viewer. Furthermore, “position_radius [o]” indicates the distance from the viewer to the O-th object.

  Therefore, the position of the object in the three-dimensional space is specified from these "position_azimuth [o]", "position_elevation [o]" and "position_radius [o]", and these pieces of information are the spatial position of the object. It is considered as information.

  Also, "gain_factor [o]" indicates the gain of the O-th object.

  Thus, the metadata shown in FIG. 15 includes “object_priority [o]”, “position_azimuth [o]”, “position_elevation [o]”, “position_radius [o]”, and “gain_factor [] for one object. o] are arranged in order as data of the object. Then, in the metadata, data of each object is arranged, for example, in the order of the object number of the object.

Fourth Embodiment
<About noise due to perfect reconstruction of audio signal and discontinuity>
In the above, when the priority information of each frame (hereinafter particularly referred to as a time frame) for each channel or object read from the bit stream in the decoding device 151 is less than a predetermined threshold value, IMDCT etc. The example which reduces the processing amount at the time of decoding was demonstrated by omitting the decoding process of. Specifically, it has been described that when the priority information is less than the threshold, the zero value output unit 195 and the zero value output unit 200 output a silent audio signal, that is, output zero data as an audio signal.

  However, in such a case, the sound quality on the auditory sense deteriorates. Specifically, the sound quality deterioration occurs due to the sound quality deterioration due to the complete reconstruction of the audio signal and the noise generation due to the signal discontinuity such as the glitch noise.

(Deterioration in sound quality due to complete reconstruction)
For example, when 0 data is output as an audio signal when the priority information is less than the threshold value, sound quality deterioration occurs when switching between 0 data output and a normal audio signal output that is not 0 data.

  As described above, in the unpacking / decoding unit 161, the IMDCT unit 196 and the IMDCT unit 201 perform IMDCT on MDCT coefficients for each time frame read from the bit stream. Then, in more detail, the unpacking / decoding unit 161 generates an audio signal of the current time frame from the result of IMDCT or 0 data of the current time frame and the result of IMDCT one time frame before or 0 data. Ru.

  Here, generation of an audio signal will be described with reference to FIG. Here, although generation of an audio signal of an object is described as an example, the same applies to generation of an audio signal of each channel. Also, in the following, the audio signal output from the zero value output unit 200 and the audio signal output from the IMDCT unit 201 will be particularly referred to as an IMDCT signal. Similarly, the audio signal output from the zero value output unit 195 and the audio signal output from the IMDCT unit 196 will be particularly referred to as an IMDCT signal.

  In FIG. 16, the horizontal direction in the figure indicates time, and the rectangles in which the characters "data [n-1]" to "data [n + 2]" are written are the time frames of the predetermined object (n It represents a bitstream from -1) to a time frame (n + 2). Also, the numerical value in the bit stream of each time frame indicates the value of the priority information of the object of that time frame, and in this example, the value of the priority information of each time frame is “7”.

  Furthermore, the rectangles in which the characters “MDCT_coef [q]” (where q = n−1, n,...) Are written in FIG. 16 respectively represent MDCT coefficients of the time frame (q).

  Now, assuming that the threshold Q = 4, the value “7” of the priority information of the time frame (n−1) is equal to or greater than the threshold Q, so IMDCT is performed on the MDCT coefficients for the time frame (n−1). Is done. Similarly, since the value “7” of the priority information of time frame (n) is also greater than or equal to the threshold value Q, IMDCT is performed on the MDCT coefficients for time frame (n).

  As a result, it is assumed that the IMDCT signal OPS11 of time frame (n-1) and the IMDCT signal OPS12 of time frame (n) are obtained.

  In this case, the unpacking / decoding unit 161 adds the first half portion of the IMDCT signal OPS12 of the time frame (n) and the second half portion of the IMDCT signal OPS11 of the time frame (n-1) one time frame earlier An audio signal of a time frame (n), that is, an audio signal of a period FL (n) is used. In other words, the portion of the IMDCT signal OPS11 for the period FL (n) and the portion of the IMDCT signal OPS12 for the period FL (n) are overlap-added, and the time frame (n) before coding of the object to be processed Audio signal is reproduced.

  Such processing is processing required to completely reconstruct the IMDCT signal into a signal before MDCT.

  However, in the above-mentioned unpacking / decoding unit 161, for example, as shown in FIG. 17, according to the priority information of each time frame, at timing when the IMDCT signal of IMDCT unit 201 and the IMDCT signal of zero value output unit 200 are switched. , IMDCT signals are not completely reconstructed into signals before MDCT. That is, if 0 data is used instead of the original signal at the time of overlap addition, the original audio signal can not be reproduced because the signal is not completely reconstructed, and the audible sound quality of the audio signal is degraded. .

  Note that, in FIG. 17, the same characters and the like are described in portions corresponding to the case in FIG. 16, and the description thereof is omitted.

  In the example of FIG. 17, although the value of the priority information of the time frame (n−1) is “7”, the priority information of the other time frame (n) to the time frame (n + 2) is the lowest “ It is 0.

  Therefore, assuming that the threshold Q = 4, IMDCT is performed on the MDCT coefficients in the IMDCT unit 201 for the time frame (n−1), and the IMDCT signal OPS21 of the time frame (n−1) is obtained. On the other hand, no IMDCT is performed on the MDCT coefficients for the time frame (n), and 0 data output from the 0-value output unit 200 is used as the IMDCT signal OPS22 of the time frame (n).

  In this case, the first half portion of 0 data which is IMDCT signal OPS22 of time frame (n) and the second half portion of IMDCT signal OPS 21 of the time frame (n-1) one time frame before are added together It is an audio signal of time frame (n). That is, the portions of the IMDCT signal OPS22 and the IMDCT signal OPS21 during the period FL (n) are overlap-added to be an audio signal of the final time frame (n) of the object to be processed.

  Thus, when the output source of the IMDCT signal switches from the IMDCT unit 201 to the 0 value output unit 200 or from the 0 value output unit 200 to the IMDCT unit 201, the IMDCT signal from the IMDCT unit 201 is not completely reconstructed. Deterioration of the sound quality on the sense of hearing occurs.

(Deterioration of sound quality due to noise generation caused by discontinuity)
In addition, when the output source of the IMDCT signal is switched from the IMDCT unit 201 to the zero value output unit 200 or from the zero value output unit 200 to the IMDCT unit 201, the signal is not completely reconstructed, so the IMDCT signal obtained by IMDCT In some cases, the signal may be discontinuous at the connection portion of the IMDCT signal, which is zero data. Then, glitch noise occurs at the discontinuous connection portion, and the audio quality of the audio signal deteriorates.

  Furthermore, in order to improve the sound quality in the unpacking / decoding unit 161, SBR (Spectral Band) is applied to the audio signal obtained by overlap addition of the IMDCT signals output from the IMDCT unit 201 or the zero value output unit 200. Processing such as Replication may be performed.

  Although various processes can be considered as processes of the latter stage of the IMDCT unit 201 and the zero value output unit 200, the following description will be made with SBR as an example.

  In SBR, the high frequency component of the original audio signal before encoding is generated from the low frequency component audio signal obtained by overlap addition and the high frequency power value stored in the bit stream Be done.

  Specifically, an audio signal of one time frame is divided into several sections called time slots, and the audio signal of each time slot is a signal of a plurality of low bands (hereinafter referred to as low band sub band signals) Also divided into two).

  Then, based on the low band sub band signal of each sub band and the power value of each high band sub band, a signal of each high band sub band (hereinafter also referred to as high band sub band signal) is generated . For example, the target high band sub-band signal is generated by performing power adjustment or frequency shift of the low band sub-band signal of a predetermined sub-band according to the power value of the target sub-band of the high band. .

  Further, the high band sub-band signal and the low band sub-band signal are combined to generate an audio signal including a high band component, and an audio signal including a high band component generated for each time slot is combined to form a high band It is an audio signal of one time frame including components.

  When such SBR is performed in the subsequent stage of the IMDCT unit 201 or the zero value output unit 200, the high frequency component is generated by the SBR for an audio signal composed of the IMDCT signal output from the IMDCT unit 201. However, since the IMDCT signal output from the 0 value output unit 200 is 0 data, the high frequency component obtained by SBR is also 0 data for an audio signal composed of the IMDCT signal output from the 0 value output unit 200. It will

  Then, when the output source of the IMDCT signal is switched from the IMDCT unit 201 to the zero value output unit 200 or from the zero value output unit 200 to the IMDCT unit 201, the connection portion is also discontinuous in the high frequency band. There is. In such a case, glitch noise occurs and the audible sound quality is degraded.

  Therefore, in the present technology, selection of the output destination of MDCT coefficients taking into consideration previous and subsequent time frames, and fade-in processing and fade-out processing on an audio signal are performed to suppress the above-described deterioration in audio quality and improve sound quality. I did it.

<Selection of MDCT coefficient output destination in consideration of preceding and following time frames>
First, the selection of the output destination of the MDCT coefficients in consideration of the previous and subsequent time frames will be described. Here, although the audio signal of the object is described as an example, the same applies to the audio signal of each channel. Also, the processing described below is performed for each object and for each channel.

  For example, in the above-described embodiment, it has been described that the output selection unit 199 selectively switches the output destination of the MDCT coefficient of each object based on the priority information of the current time frame. On the other hand, in the present embodiment, the output selection unit 199 selects one of the current time frame, a time frame immediately before the current time frame, and three consecutive time frames after the current time frame. Switch the output destination of MDCT coefficients based on priority information of one time frame. In other words, whether or not to decode encoded data is selected based on priority information of three consecutive time frames.

  Specifically, when the conditional expression shown in the following equation (1) is satisfied for the object to be processed, the output selection unit 199 supplies the MDCT coefficients of the time frame (n) of the object to the IMDCT unit 201.

  In equation (1), object_priority [q] (where q = n -1, n, n + 1) indicates the priority information of each time frame (q), and thre indicates the threshold Q.

  Therefore, if there is a time frame in which at least one of the priority information has a threshold value Q or more in three continuous time frames in total including the current time frame and the time frames before and after the current time frame, the MDCT coefficient is supplied to The IMDCT unit 201 is selected. In this case, decoding of coded data, more specifically, IMDCT on MDCT coefficients is performed. On the other hand, if all the priority information of these three time frames is less than the threshold Q, the MDCT coefficient is set to 0 and output to the 0 value output unit 200. In this case, decoding of coded data, more specifically, IMDCT on MDCT coefficients is not substantially performed.

  As a result, as shown in FIG. 18, the audio signal is completely reconstructed from the IMDCT signal, and the deterioration of the audible sound quality is suppressed. Note that, in FIG. 18, the same characters and the like are described in portions corresponding to the case in FIG. 16, and the description thereof is omitted.

  In the example shown in the upper side of FIG. 18, the value of the priority information of each time frame is the same as the example shown in FIG. For example, assuming that the threshold Q = 4, in the example shown on the upper side in the figure, the priority information of the time frame (n-1) is equal to or higher than the threshold Q, but the time frame (n) to the time frame (n + 2) Then, the priority information is less than the threshold Q.

  Therefore, IMDCT is performed on the MDCT coefficients of time frame (n-1) and time frame (n) from the conditional expression shown in equation (1) to obtain IMDCT signal OPS31 and IMDCT signal OPS32. On the other hand, in the time frame (n + 1) in which the conditional expression is not satisfied, IMDCT is not performed on the MDCT coefficients, and 0 data is set as the IMDCT signal OPS33.

  Therefore, the audio signal of the time frame (n) which is not completely reconstructed in the example of FIG. 17 becomes completely reconstructed in the example shown at the upper side of FIG. 18, and the deterioration of the audible sound quality is suppressed. . However, in this example, since the audio signal is not completely reconstructed in the next time frame (n + 1), fade-out processing to be described later is performed in time frame (n) and time frame (n + 1). Deterioration of the sound quality of is suppressed.

  Further, in the example shown on the lower side in the figure, the priority information is less than the threshold Q in the time frame (n-1) to the time frame (n + 1), and the priority in the time frame (n + 2) The information is above the threshold Q.

  Therefore, according to the conditional expression shown in equation (1), IMDCT is not performed on the MDCT coefficients in the time frame (n) in which the conditional expression is not satisfied, and 0 data is set as the IMDCT signal OPS41. On the other hand, IMDCT is performed on the MDCT coefficients of time frame (n + 1) and time frame (n + 2) to obtain IMDCT signal OPS 42 and IMDCT signal OPS 43, respectively.

  In this example, the audio signal can be completely reconstructed in the time frame (n + 2) at which the priority information switches from a value less than the threshold Q to a value greater than or equal to the threshold Q. Can be suppressed. However, even in this case, since the audio signal is not completely reconstructed in the immediately preceding time frame (n + 1), fade-in processing to be described later is performed in time frame (n + 1) and time frame (n + 2). And the deterioration of the sound quality on hearing is suppressed.

  Here, the priority information is pre-read for one hour frame, and the output destination of the MDCT coefficient is selected from the priority information of the continuous three-hour frame. Therefore, fade-out processing is performed in time frame (n) and time frame (n + 1) in the example shown at the upper side in the figure, and time frame (n + 1) and time in the example shown at the lower side in the figure. Fade-in processing is performed at frame (n + 2).

  However, in the case where priority information of 2 hour frames can be read ahead, fade-out processing is performed in time frame (n + 1) and time frame (n + 2) in the example shown at the upper side in the figure. The fade-in process may be performed in the time frame (n) and the time frame (n + 1) in the example shown on the lower side in the figure.

<Fade in process and fade out process>
Next, fade-in processing and fade-out processing for an audio signal will be described. Here, although the audio signal of the object is described as an example, the same applies to the audio signal of each channel. Also, fade-in processing and fade-out processing are performed for each object and for each channel.

  In the present technology, for example, as in the example illustrated in FIG. 18, fade-in occurs in a time frame in which IMDCT signal obtained by IMDCT and IMDCT signal that is 0 data are overlap-added and before or after that. Processing or fade-out processing is performed.

  In the fade-in process, gain adjustment is performed on the audio signal so that the amplitude (magnitude) of the audio signal in that time frame increases with time. Conversely, in the fade-out process, gain adjustment is performed on the audio signal so that the amplitude of the audio signal in that time frame decreases with time.

  As a result, even when the connection portion between the IMDCT signal obtained by IMDCT and the IMDCT signal converted into 0 data is discontinuous, it is possible to suppress the deterioration of the sound quality on the audibility. Hereinafter, the gain value to be multiplied by the audio signal at the time of such gain adjustment will be particularly referred to as fading signal gain.

  Furthermore, in the present technology, fade-in processing or fade-out processing is performed also in SBR for the connection portion between the IMDCT signal obtained by IMDCT and the IMDCT signal that is 0 data.

  That is, in SBR, the power value of each high band is used for each time slot, but in the present technology, the gain value determined for each time slot for fade-in processing or fade-out processing is each high band The power values of the subbands are multiplied to perform SBR. That is, gain adjustment of the power value of the high frequency band is performed.

  Hereinafter, the gain value determined for each time slot, which is multiplied by the power value of the high frequency band, will be particularly referred to as a fading SBR gain.

  Specifically, the fading SBR gain for fade-in processing is determined such that the gain value becomes larger with time, that is, the value becomes larger as the fading SBR gain of the time slot behind in time It is done. Conversely, the fading SBR gain for fade-out processing is determined so that the value becomes smaller as the fading SBR gain of the time slot behind in time.

  As described above, by performing the fade-in process and the fade-out process also at the time of SBR, it is possible to suppress the deterioration of the sound quality in the sense of hearing even when the high region becomes discontinuous.

  Specifically, for example, the processes shown in FIG. 19 and FIG. 20 are performed as gain adjustment such as fade-in processing and fade-out processing for the audio signal and the power value of the high band. In FIGS. 19 and 20, the portions corresponding to the case in FIG. 18 have the same characters, reference numerals, etc., and the description thereof will be omitted.

  The example shown in FIG. 19 is an example shown in the upper side in FIG. In this example, the audio signal of the time frame (n) and the time frame (n + 1) is multiplied by the fading signal gain indicated by the polygonal line GN11.

  The value of the fading signal gain indicated by the broken line GN11 linearly changes from "1" to "0" with time in the time frame (n) portion, and continuously in the time frame (n + 1) portion. It is 0. Therefore, the audio signal gradually changes to zero data by the gain adjustment of the audio signal by the fading signal gain, so that it is possible to suppress the deterioration of the sound quality on the audibility.

  Also, in this example, the high frequency power value of each time slot of the time frame (n) is multiplied by the fading SBR gain indicated by the arrow GN12.

  The value of the fading SBR gain indicated by the arrow GN12 changes from “1” to “0” so as to decrease as time goes backward in time. Therefore, the high frequency band component of the audio signal gradually changes to zero data by the gain adjustment of the high frequency band by the fading SBR gain, so that it is possible to suppress the deterioration of the sound quality on the audibility.

  On the other hand, the example shown in FIG. 20 is an example shown in the lower side in the drawing of FIG. In this example, the audio signal of the time frame (n + 1) and the time frame (n + 2) is multiplied by the fading signal gain indicated by the polygonal line GN21.

  The value of the fading signal gain indicated by the broken line GN21 continues to be “0” in the part of the time frame (n + 1), and from “0” with time in the part of the time frame (n + 2) It changes linearly up to "1". Therefore, the audio signal gradually changes from the zero data to the original signal by the gain adjustment of the audio signal by the fading signal gain, so that it is possible to suppress the deterioration of the sound quality on the audibility.

  Further, in this example, the high frequency power value of each time slot of the time frame (n + 2) is multiplied by the fading SBR gain indicated by the arrow GN22.

  The value of the fading SBR gain indicated by the arrow GN 22 changes from “0” to “1” so as to increase as time goes backward in time. Therefore, since the high frequency component of the audio signal gradually changes from the zero data to the original signal by the gain adjustment of the high frequency by the fading SBR gain, it is possible to suppress the deterioration of the audible sound quality.

<Configuration Example of Unpacking / Decoding Unit>
In the case where the selection of the output destination of the MDCT coefficient described above and the gain adjustment such as fade-in processing or fade-out processing are performed, the unpacking / decoding unit 161 is configured, for example, as shown in FIG. In FIG. 21, parts corresponding to those in FIG. 10 are assigned the same reference numerals, and the description thereof will be omitted as appropriate.

  The unpacking / decoding unit 161 shown in FIG. 21 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, overlap addition. Unit 271, gain adjustment unit 272, SBR processing unit 273, object audio signal acquisition unit 197, object audio signal decoding unit 198, output selection unit 199, zero value output unit 200, IMDCT unit 201, overlap addition unit 274, gain adjustment And a SBR processing unit 276.

  The configuration of the unpacking / decoding unit 161 shown in FIG. 21 is a configuration in which the overlap adding unit 271 to the SBR processing unit 276 are further provided to the configuration of the unpacking / decoding unit 161 shown in FIG.

  The overlap addition unit 271 generates an audio signal of each time frame by performing overlap addition on the IMDCT signal (audio signal) supplied from the zero value output unit 195 or the IMDCT unit 196, and supplies the audio signal to the gain adjustment unit 272. Do.

  The gain adjustment unit 272 adjusts the gain of the audio signal supplied from the overlap addition unit 271 based on the priority information supplied from the priority information acquisition unit 191, and supplies the audio signal to the SBR processing unit 273.

  The SBR processing unit 273 acquires the power value of each high band sub-band for each time slot from the priority information acquisition unit 191 and, based on the priority information supplied from the priority information acquisition unit 191, Adjust the power value gain. Also, the SBR processing unit 273 performs SBR on the audio signal supplied from the gain adjusting unit 272 using the gain-adjusted power value of the high band, and the audio signal obtained as a result is output to the mixing unit 163. Supply.

  The overlap addition unit 274 generates an audio signal of each time frame by performing overlap addition on the IMDCT signal (audio signal) supplied from the zero value output unit 200 or the IMDCT unit 201, and supplies the audio signal to the gain adjustment unit 275. Do.

  The gain adjustment unit 275 adjusts the gain of the audio signal supplied from the overlap addition unit 274 based on the priority information supplied from the priority information acquisition unit 191, and supplies the audio signal to the SBR processing unit 276.

  The SBR processing unit 276 acquires the power value of each high band sub-band for each time slot from the priority information acquisition unit 191, and based on the priority information supplied from the priority information acquisition unit 191, Adjust the power value gain. In addition, the SBR processing unit 276 performs SBR on the audio signal supplied from the gain adjusting unit 275 using the gain-adjusted power value of the high band, and outputs the audio signal obtained as a result to the rendering unit 162. Supply.

<Description of selective decoding process>
Subsequently, an operation of the decoding device 151 in the case where the unpacking / decoding unit 161 is configured as shown in FIG. 21 will be described. In this case, the decoding device 151 performs the decoding process described with reference to FIG. However, the process shown in FIG. 22 is performed as the selective decoding process of step S52.

  The selective decoding process corresponding to the process of step S52 in FIG. 11 will be described below with reference to the flowchart in FIG.

  In step S181, the priority information acquisition unit 191 acquires the power value of the high frequency band of the audio signal of each channel from the supplied bit stream and supplies the power value to the SBR processing unit 273. The high frequency power value of the audio signal is acquired and supplied to the SBR processing unit 276.

  When the power value of the high frequency band is acquired, the processing of steps S182 to S187 is performed thereafter to generate the audio signal (IMDCT signal) of the channel to be processed. Since the process is the same as the process of step S86, the description thereof is omitted.

  However, in step S186, if the same conditional expression as the above-mentioned expression (1) is satisfied, that is, the priority information of the current time frame of the channel to be processed and each time frame immediately before and after the current time frame. If at least one of the priority information is equal to or higher than the threshold P, the priority information is determined to be equal to or higher than the threshold P. Further, the IMDCT signal generated by the zero value output unit 195 or the IMDCT unit 196 is output to the overlap addition unit 271.

  If it is determined in step S186 that the priority information is not equal to or higher than the threshold value P, or if the IMDCT signal is generated in step S187, the process of step S188 is performed.

  In step S188, the overlap addition unit 271 performs overlap addition of the IMDCT signals supplied from the zero value output unit 195 or the IMDCT unit 196, and the audio signal of the current time frame obtained as a result is supplied to the gain adjustment unit 272. Supply.

  Specifically, for example, as described with reference to FIG. 18, the first half portion of the IMDCT signal of the current time frame and the second half portion of the IMDCT signal of the immediately preceding time frame are added together to generate the audio signal of the current time frame. Be done.

  In step S189, the gain adjustment unit 272 performs gain adjustment on the audio signal supplied from the overlap addition unit 271 based on the priority information of the processing target channel supplied from the priority information acquisition unit 191, and performs SBR processing. It supplies to the part 273.

  Specifically, the gain adjustment unit 272 determines that the priority information of the time frame immediately before the current time frame is the threshold P or more, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame If the information is less than the threshold value P, the gain of the audio signal is adjusted with the fading signal gain indicated by the broken line GN11 in FIG. In this case, the time frame (n) in FIG. 19 corresponds to the current time frame, and in the time frame immediately after the current time frame, gain adjustment with fading signal gain = 0 is performed as shown by the polygonal line GN11. .

  In addition, when the priority information of the current time frame is equal to or higher than the threshold P and the priority information of the two-hour frame immediately before the current time frame is less than the threshold P, the gain adjustment unit 272 applies the line GN21 in FIG. Adjust the gain of the audio signal with the fading signal gain shown. In this case, the time frame (n + 2) in FIG. 20 corresponds to the current time frame, and in the time frame immediately before the current time frame, gain adjustment with fading signal gain = 0 is shown as shown by the polygonal line GN21. To be done.

  Note that the gain adjustment unit 272 performs gain adjustment only in the two examples, and otherwise does not perform gain adjustment, and supplies the audio signal to the SBR processing unit 273 as it is.

  In step S190, the SBR processing unit 273 sets the audio signal supplied from the gain adjustment unit 272 based on the power value of the high band of the processing target channel and the priority information supplied from the priority information acquisition unit 191. Perform SBR against.

  Specifically, the SBR processing unit 273 determines that the priority information of the time frame immediately before the current time frame is equal to or more than the threshold P, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame. If the degree information is less than the threshold value P, the power value of the high region is gain-adjusted by the fading SBR gain indicated by the arrow GN12 in FIG. That is, the high band power value is multiplied by the fading SBR gain.

  Then, the SBR processing unit 273 performs SBR using the gain-adjusted high frequency power value, and supplies the audio signal obtained as a result to the mixing unit 163. In this case, the time frame (n) in FIG. 19 corresponds to the current time frame.

  If the priority information of the current time frame is equal to or greater than the threshold P and the priority information of the two-hour frame immediately before the current time frame is less than the threshold P, the SBR processing unit 273 causes an arrow GN22 in FIG. Gain adjust the high band power value with the indicated fading SBR gain. Then, the SBR processing unit 273 performs SBR using the gain-adjusted high frequency power value, and supplies the audio signal obtained as a result to the mixing unit 163. In this case, the time frame (n + 2) in FIG. 20 corresponds to the current time frame.

  In addition, the SBR processing unit 273 performs gain adjustment of the high band power value only in the case of these two examples, and otherwise performs the gain adjustment without changing the acquired high band power value as it is. Using SBR, the audio signal obtained as a result is supplied to the mixing unit 163.

  After the SBR is performed and the audio signal of the current time frame is obtained, the processes of steps S191 to S196 are performed, and since these processes are similar to the processes of steps S87 to S92 of FIG. The explanation is omitted.

  However, in step S195, when the conditional expression of the above-mentioned expression (1) is satisfied, it is determined that the priority information is equal to or higher than the threshold value Q. Further, the IMDCT signal (audio signal) generated by the zero value output unit 200 or the IMDCT unit 201 is output to the overlap addition unit 274.

  Thus, when the IMDCT signal of the current time frame is obtained, the processes of steps S197 to S199 are performed to generate the audio signal of the current time frame, but these processes are the same as the processes of steps S188 to S190. The description is omitted because it is similar.

  In step S200, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S193. Then, if it is determined in step S193 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S53 in FIG.

  As described above, the unpacking / decoding unit 161 selects the output destination of the MDCT coefficient according to the priority information of the current time frame and the time frames before and after the current time frame. As a result, the audio signal is completely reconstructed in the switching portion between the time frame in which the priority information is equal to or higher than the threshold and the time frame in which the priority information is less than the threshold, thereby suppressing the deterioration in auditory sound quality. can do.

  In addition, the unpacking / decoding unit 161 adjusts the gain of the audio signal after overlap addition and the power value of the high band based on the priority information of three consecutive time frames. That is, fade-in processing and fade-out processing are appropriately performed. As a result, it is possible to suppress the occurrence of glitch noise and to suppress the deterioration of the sound quality on hearing.

Fifth Embodiment
<Fade in process and fade out process>
In the fourth embodiment, it has been described that the gain adjustment is performed on the audio signal after the overlap addition, and the gain adjustment on the high band power value is performed in the SBR. In this case, gain adjustment, that is, fade-in processing or fade-out processing is separately performed on the low-pass component and high-pass component of the final audio signal.

  Therefore, the gain adjustment is performed on the audio signal obtained by SBR without performing the gain adjustment immediately after overlap addition and at SBR so that these fade-in processing and fade-out processing can be realized with less processing. You may do so.

  In such a case, gain adjustment is performed as shown in FIG. 23 and FIG. 24, for example. In FIGS. 23 and 24, the same characters and the like are described in the parts corresponding to those in FIGS. 19 and 20, and the description thereof is omitted.

  The example shown in FIG. 23 is an example in which the change of the priority information is the same as the case shown in FIG. In this example, assuming that the threshold Q = 4, the priority information of the time frame (n-1) is equal to or higher than the threshold Q, but in the time frame (n) to the time frame (n + 2), the priority information Is less than the threshold Q.

  In such a case, the audio signal obtained by SBR in the time frame (n) and the time frame (n + 1) is multiplied by the fading signal gain indicated by the polygonal line GN 31 to be gain adjusted. Become.

  The fading signal gain indicated by the broken line GN31 is the same as the fading signal gain indicated by the broken line GN11 in FIG. However, in the case of the example of FIG. 23, since the audio signal to be subjected to the gain adjustment includes both the low frequency component and the high frequency component, the gains of the low frequency component and the high frequency component are included. The adjustment can be done with one fading signal gain.

  By adjusting the gain of the audio signal with such a fading signal gain, the audio signal is gradually added to the portion where the IMDCT signal obtained by IMDCT and the IMDCT signal made zero data are overlap-added and the portion immediately before that. Will change to 0 data. Thereby, it is possible to suppress the deterioration of the sound quality on the audibility.

  On the other hand, the example shown in FIG. 24 is an example in which the change of the priority information is the same as the case shown in FIG. In this example, assuming that the threshold Q = 4, in the time frame (n) and the time frame (n + 1), the priority information is less than the threshold Q, but the priority information of the time frame (n + 2) is It is above the threshold Q.

  In such a case, the audio signal obtained by SBR in the time frame (n + 1) and the time frame (n + 2) is multiplied by the fading signal gain indicated by the polygonal line GN 41 to perform gain adjustment It will be.

  The fading signal gain indicated by the broken line GN41 is the same as the fading signal gain indicated by the broken line GN21 in FIG. However, in the case of the example of FIG. 24, since the audio signal to be subjected to the gain adjustment includes both the low frequency component and the high frequency component, the gains of the low frequency component and the high frequency component are included. The adjustment can be done with one fading signal gain.

  By adjusting the gain of the audio signal with such a fading signal gain, the audio signal is 0 in the portion where the IMDCT signal obtained by IMDCT and the IMDCT signal converted to 0 data are subjected to overlap addition and the portion immediately thereafter. It will gradually change from data to the original signal. Thereby, it is possible to suppress the deterioration of the sound quality on the audibility.

<Configuration Example of Unpacking / Decoding Unit>
When the gain adjustment by the fade-in process and the fade-out process described with reference to FIGS. 23 and 24 is performed, the unpacking / decoding unit 161 is configured, for example, as illustrated in FIG. In FIG. 25, portions corresponding to the case in FIG. 21 are denoted with the same reference numerals, and the description thereof will be appropriately omitted.

  The unpacking / decoding unit 161 shown in FIG. 25 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, overlap addition. Unit 271, SBR processing unit 273, gain adjustment unit 272, object audio signal acquisition unit 197, object audio signal decoding unit 198, output selection unit 199, zero value output unit 200, IMDCT unit 201, overlap addition unit 274, SBR processing And a gain adjustment unit 275.

  The configuration of the unpacking / decoding unit 161 shown in FIG. 25 is shown in FIG. 21 in that the gain adjusting unit 272 and the gain adjusting unit 275 are respectively disposed after the SBR processing unit 273 and the SBR processing unit 276. This differs from the configuration of the unpacking / decoding unit 161.

  In the unpacking / decoding unit 161 shown in FIG. 25, the SBR processing unit 273 performs processing on the audio signal supplied from the overlap addition unit 271 based on the high frequency power value supplied from the priority information acquisition unit 191. Then, SBR is performed, and the audio signal obtained as a result is supplied to the gain adjustment unit 272. In this case, the SBR processing unit 273 does not perform gain adjustment of the high frequency power value.

  The gain adjustment unit 272 adjusts the gain of the audio signal supplied from the SBR processing unit 273 based on the priority information supplied from the priority information acquisition unit 191, and supplies the audio signal to the mixing unit 163.

  The SBR processing unit 276 performs SBR on the audio signal supplied from the overlap addition unit 274 based on the high frequency power value supplied from the priority information acquisition unit 191, and the audio signal obtained as a result thereof. Are supplied to the gain adjustment unit 275. In this case, the SBR processing unit 276 does not perform gain adjustment of the high frequency power value.

  The gain adjustment unit 275 adjusts the gain of the audio signal supplied from the SBR processing unit 276 based on the priority information supplied from the priority information acquisition unit 191, and supplies the audio signal to the rendering unit 162.

<Description of selective decoding process>
Subsequently, an operation of the decoding device 151 in the case where the unpacking / decoding unit 161 is configured as shown in FIG. 25 will be described. In this case, the decoding device 151 performs the decoding process described with reference to FIG. However, the process shown in FIG. 26 is performed as the selective decoding process of step S52.

  The selective decoding process corresponding to the process of step S52 of FIG. 11 will be described below with reference to the flowchart of FIG. In addition, since the process of step S231 thru | or step S238 is the same as the process of step S181 thru | or step S188 of FIG. 22, the description is abbreviate | omitted. However, in step S232, priority information is not supplied to the SBR processing unit 273 and the SBR processing unit 276.

  In step S239, the SBR processing unit 273 performs SBR on the audio signal supplied from the overlap addition unit 271 based on the high frequency power value supplied from the priority information acquisition unit 191, and the result is obtained as a result. The supplied audio signal is supplied to the gain adjustment unit 272.

  In step S 240, the gain adjustment unit 272 performs gain adjustment on the audio signal supplied from the SBR processing unit 273 based on the priority information of the processing target channel supplied from the priority information acquisition unit 191, and the mixing unit 163. Supply to

  Specifically, the gain adjustment unit 272 determines that the priority information of the time frame immediately before the current time frame is the threshold P or more, and the priority information of the current time frame and the priority of the time frame immediately after the current time frame If the information is less than the threshold value P, the gain of the audio signal is adjusted with the fading signal gain indicated by the polygonal line GN 31 in FIG. In this case, the time frame (n) in FIG. 23 corresponds to the current time frame, and in the time frame immediately after the current time frame, gain adjustment with fading signal gain = 0 is performed as indicated by the polygonal line GN31. .

  In addition, when the priority information of the current time frame is equal to or higher than the threshold P and the priority information of the two-hour frame immediately before the current time frame is less than the threshold P, the gain adjustment unit 272 causes the broken line GN41 in FIG. Adjust the gain of the audio signal with the fading signal gain shown. In this case, the time frame (n + 2) in FIG. 24 corresponds to the current time frame, and in the time frame immediately before the current time frame, gain adjustment with fading signal gain = 0 is shown as shown by the polygonal line GN41. To be done.

  The gain adjustment unit 272 performs gain adjustment only in the two examples, and supplies the audio signal as it is to the mixing unit 163 without performing gain adjustment in other cases.

  After the gain adjustment of the audio signal is performed, the processes of steps S241 to S247 are performed, but these processes are the same as the processes of steps S191 to S197 of FIG.

  Thus, when the audio signal of the current time frame of the object to be processed is obtained, the processes of steps S248 and S249 are performed to generate the audio signal of the final current time frame. Since the process is the same as the process of step S239 and step S240, the description thereof is omitted.

  In step S250, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S243. Then, if it is determined in step S243 that the object number is not less than N, the selective decoding process ends, and the process then proceeds to step S53 in FIG.

  As described above, the unpacking / decoding unit 161 adjusts the gain of the audio signal obtained by the SBR based on the priority information of consecutive three time frames. As a result, it is possible to more easily suppress the occurrence of glitch noise and to suppress the deterioration of the sound quality on hearing.

  In this embodiment, the selection of the output destination of the MDCT coefficient using priority information for three time frames and the gain adjustment by the fading signal gain have been described, but the gain by the fading signal gain is described. Only adjustment may be performed.

  In such a case, the output selection unit 194 and the output selection unit 199 select the output destination of the MDCT coefficient by the same process as in the first embodiment. Then, when the priority information of the current time frame is less than the threshold, the gain adjusting unit 272 and the gain adjusting unit 275 linearly increase or decrease the fading signal gain of the current time frame to perform fade-in processing or fade-out. Do the processing. Here, whether to perform the fade-in process or the fade-out process may be determined from the priority information of the current time frame and the priority information of the time frames before and after that.

Sixth Embodiment
<Fade in process and fade out process>
By the way, in the rendering unit 162, for example, VBAP is performed, and an audio signal of each channel for reproducing the sound of each object is generated from the audio signal of each object.

  Specifically, in the VBAP, the gain value of the audio signal (hereinafter also referred to as the VBAP gain) is calculated for each object for each channel, that is, for each speaker that outputs audio. Then, the sum of the audio signal of each object multiplied by the VBAP gain for the same channel (speaker) is taken as the audio signal of that channel. In other words, for each object, the audio signal of the object is assigned to each of those channels with the VBAP gain calculated for each channel.

  Therefore, with regard to the audio signal of the object, it is not necessary to adjust the gain of the audio signal of the object or the power value of the high region, but by adjusting the VBAP gain appropriately, generation of glitch noise is suppressed to improve audio quality. It is also possible to suppress the deterioration of the

  In such a case, for example, linear interpolation or the like is performed on the VBAP gain of each time frame, the VBAP gain for each sample of the audio signal in each time frame is calculated, and the audio signal of each channel is calculated by the obtained VBAP gain. Is generated.

  For example, the value of the VBAP gain of the first sample of the time frame to be processed is taken as the value of the VBAP gain of the sample at the end of the time frame immediately before the time frame to be processed. Further, the value of the VBAP gain of the sample at the end of the process target time frame is taken as the value of the VBAP gain calculated by the normal VBAP for the process target time frame.

  Then, in the time frame to be processed, the value of the VBAP gain of each sample between the first sample and the last sample is determined so that the VBAP gain linearly changes from the first sample to the last sample.

  However, when the priority information of the time frame to be processed is less than the threshold, the VBAP calculation is not performed, and the value of the VBAP gain of the sample at the end of the time frame to be processed is set to zero. Then, the VBAP gain of each sample is determined so that the VBAP gain linearly changes from the first sample of the time frame to be processed to the last sample.

  By thus adjusting the gain of the audio signal of each object using the VBAP gain, the gain adjustment of the low frequency component and the high frequency component can be performed at one time, and the generation of glitch noise can be suppressed with a smaller processing amount. It is possible to suppress the deterioration of the sound quality on hearing.

  When the VBAP gain is determined for each sample as described above, the VBAP gain for each sample of each time frame is, for example, as shown in FIG. 27 and FIG.

  In FIGS. 27 and 28, parts corresponding to those in FIGS. 19 and 20 have the same characters, etc., and the description thereof will be omitted. Further, in FIGS. 27 and 28, “VBAP_gain [q] [s]” (where q = n−1, n, n + 1, n + 2) is a speaker that specifies a speaker corresponding to a predetermined channel. It shows the VBAP gain of the time frame (q) of the object to be processed, whose index is s.

  The example shown in FIG. 27 is an example in which the change of the priority information is the same as the case shown in FIG. In this example, assuming that the threshold Q = 4, the priority information of the time frame (n-1) is equal to or higher than the threshold Q, but in the time frame (n) to the time frame (n + 2), the priority information Is less than the threshold Q.

  In such a case, the VBAP gain of the time frame (n-1) to the time frame (n + 1) is, for example, the gain indicated by the broken line GN51.

  In this example, since the priority information of the time frame (n-1) is equal to or higher than the threshold value Q, the VBAP gain of each sample is determined based on the VBAP gain calculated by the normal VBAP.

  That is, the value of the VBAP gain of the first sample of time frame (n-1) is made the same as the value of the VBAP gain of the last sample of time frame (n-2). In addition, the value of the VBAP gain of the sample at the end of time frame (n-1) corresponds to the speaker s calculated by normal VBAP for time frame (n-1) for the object to be processed. It is the value of the channel VBAP gain. Then, the value of the VBAP gain of each sample in the time frame (n-1) is determined to change linearly from the leading sample to the trailing sample.

  Further, since the priority information of the time frame (n) is less than the threshold Q, the value of the VBAP gain of the sample at the end of the time frame (n) is set to zero.

  That is, the value of the VBAP gain of the sample at the beginning of time frame (n) is the same as the value of the VBAP gain of the sample at the end of time frame (n-1), and the value of VBAP gain at the end of time frame (n) The value of the gain is zero. Then, the value of the VBAP gain of each sample in the time frame (n) is determined to change linearly from the leading sample to the trailing sample.

  Furthermore, since the priority information of time frame (n + 1) is less than threshold Q, the value of the VBAP gain of the last sample of time frame (n + 1) is set to 0, resulting in time frame (n + 1). The value of the VBAP gain of all the samples in) is zero.

  Thus, by setting the value of the VBAP gain of the sample at the end of the time frame in which the priority information is less than the threshold Q to 0, the fade-out process equivalent to the example of FIG.

  On the other hand, the example shown in FIG. 28 is an example in which the change of the priority information is the same as the case shown in FIG. In this example, assuming that the threshold Q = 4, in the time frame (n-1) to the time frame (n + 1), the priority information is less than the threshold Q, but the priority of the time frame (n + 2) The information is above the threshold Q.

  In such a case, the VBAP gain of the time frame (n-1) to the time frame (n + 2) is, for example, the gain indicated by the broken line GN61.

  In this example, since the priority information of time frame (n) and the priority information of time frame (n + 1) are both less than threshold Q, the VBAP gain of all samples of time frame (n + 1) is 0 Become.

  Further, since the priority information of the time frame (n + 2) is equal to or higher than the threshold Q, based on the VBAP gain of the channel corresponding to the speaker s calculated by the normal VBAP, for the object to be processed, The VBAP gain for each sample is determined.

  That is, the value of the VBAP gain of the leading sample of time frame (n + 2) is set to 0, which is the value of the VBAP gain of the last sample of time frame (n + 1), and The value of the VBAP gain of the last sample is taken as the value of the VBAP gain calculated by the normal VBAP for the time frame (n + 2). And, the value of the VBAP gain of each sample of the time frame (n + 2) is determined to change linearly from the leading sample to the trailing sample.

  As described above, by setting the value of the VBAP gain of the sample at the end of the time frame in which the priority information is less than the threshold Q to 0, fade-in processing equivalent to the example of FIG. 24 is possible.

<Configuration Example of Unpacking / Decoding Unit>
When gain adjustment by the fade-in process and the fade-out process described with reference to FIGS. 27 and 28 is performed, the unpacking / decoding unit 161 is configured, for example, as illustrated in FIG. In FIG. 29, parts corresponding to those in FIG. 25 are assigned the same reference numerals, and the description thereof will be omitted as appropriate.

  The unpacking / decoding unit 161 shown in FIG. 29 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, overlap addition. Unit 271, SBR processing unit 273, gain adjustment unit 272, object audio signal acquisition unit 197, object audio signal decoding unit 198, output selection unit 199, zero value output unit 200, IMDCT unit 201, overlap addition unit 274, and SBR It comprises a processing unit 276.

  The configuration of the unpacking / decoding unit 161 shown in FIG. 29 is different from the configuration of the unpacking / decoding unit 161 shown in FIG. 25 in that the gain adjustment unit 275 is not provided, and the other configuration is the same. ing.

  In the unpacking / decoding unit 161 shown in FIG. 29, the SBR processing unit 276 operates on the audio signal supplied from the overlap addition unit 274 based on the high frequency power value supplied from the priority information acquisition unit 191. Then, SBR is performed, and the audio signal obtained as a result is supplied to the rendering unit 162.

  Also, the priority information acquisition unit 191 acquires metadata of each object and priority information from the supplied bit stream, and supplies the metadata to the rendering unit 162. The priority information of each object is also supplied to the output selection unit 199.

<Description of decryption processing>
Subsequently, an operation of the decoding device 151 in the case where the unpacking / decoding unit 161 is configured as shown in FIG. 29 will be described.

  In this case, the decoding device 151 performs the decoding process shown in FIG. The decoding process performed by the decoding device 151 will be described below with reference to the flowchart in FIG. However, in step S281, the same processing as the processing of step S51 in FIG. 11 is performed, and thus the description thereof is omitted.

  In step S282, the unpacking / decoding unit 161 performs selective decoding processing.

  Here, the selective decoding process corresponding to the process of step S282 of FIG. 30 will be described with reference to the flowchart of FIG.

  In addition, since the process of step S311 thru | or step S328 is the same as the process of step S231 thru | or step S248 of FIG. 26, the description is abbreviate | omitted. However, in step S312, the priority information acquisition unit 191 also supplies the priority information acquired from the bit stream to the rendering unit 162.

  In step S329, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S323. Then, if it is determined in step S323 that the object number is not less than N, the selective decoding process ends, and then the process proceeds to step S283 in FIG.

  Therefore, in the selective decoding process shown in FIG. 31, for audio signals of each channel, gain adjustment with fading signal gain is performed as in the fifth embodiment, and for each object, gain adjustment is performed. It is not performed, and the audio signal obtained by SBR is output to the rendering unit 162 as it is.

  Returning to the description of the decoding processing in FIG. 30, in step S283, the rendering unit 162 uses the audio signal of each object supplied from the SBR processing unit 276 and the metadata of each object supplied from the priority information acquisition unit 191. The audio signal of each object is rendered on the basis of the position information of and the priority information of the current time frame of each object.

  For example, as described with reference to FIG. 27 and FIG. 28, the rendering unit 162 has, for each object, priority information of the current time frame for each channel, and VBAP of a sample at the end of the time frame immediately before the current time frame. Based on the gain, the VBAP gain of each sample in the current time frame is calculated. At this time, the rendering unit 162 appropriately calculates the VBAP gain by VBAP based on the position information.

  Then, the rendering unit 162 generates an audio signal of each channel based on the VBAP gain for each sample of each channel calculated for each object and the audio signal of each object, and supplies the audio signal to the mixing unit 163.

  Although the example in which the VBAP gain of each sample is calculated so that the VBAP gain of each sample in a time frame changes linearly is described here, the VBAP gain may be changed non-linearly. Also, although an example has been described where an audio signal of each channel is generated by VBAP, even when an audio signal of each channel is generated by another method, the gain of the audio signal of each object is processed by the same processing as in VBAP. It is possible to adjust

  After the audio signal of each channel is generated, the process of step S284 is performed and the decoding process ends, but the process of step S284 is the same as the process of step S54 of FIG. Do.

  Thus, the decoding device 151 calculates the VBAP gain for each sample based on the priority information for each object, and adjusts the gain of the audio signal of the object with the VBAP gain when generating the audio signal of each channel. As a result, it is possible to suppress the occurrence of glitch noise with a smaller amount of processing, and to suppress the deterioration of the sound quality on hearing.

  In the fourth to sixth embodiments, the output destination of the MDCT coefficient is selected using the priority information of the time frame immediately before and after the current time frame, the fading signal gain, etc. Explained that the gain adjustment by. However, the present invention is not limited to this, and the priority information of the current time frame, the priority information of the time frame before the predetermined time frame of the current time frame, and the priority information of the time frame after the predetermined time frame of the current time frame And may be used.

  By the way, the series of processes described above can be executed by hardware or software. When the series of processes are performed by software, a program that configures the software is installed on a computer. Here, the computer includes, for example, a general-purpose computer capable of executing various functions by installing a computer incorporated in dedicated hardware and various programs.

  FIG. 32 is a block diagram showing a configuration example of hardware of a computer that executes the series of processes described above according to a program.

  In the computer, a central processing unit (CPU) 501, a read only memory (ROM) 502, and a random access memory (RAM) 503 are mutually connected by a bus 504.

  Further, an input / output interface 505 is connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

  The input unit 506 includes a keyboard, a mouse, a microphone, an imaging device, and the like. The output unit 507 includes a display, a speaker, and the like. The recording unit 508 includes a hard disk, a non-volatile memory, and the like. The communication unit 509 is formed of a network interface or the like. The drive 510 drives removable media 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 501 loads, for example, the program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504, and executes the above-described series. Processing is performed.

  The program executed by the computer (CPU 501) can be provided by being recorded on, for example, a removable medium 511 as a package medium or the like. Also, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the recording unit 508 via the input / output interface 505 by attaching the removable media 511 to the drive 510. Also, the program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. In addition, the program can be installed in advance in the ROM 502 or the recording unit 508.

  Note that the program executed by the computer may be a program that performs processing in chronological order according to the order described in this specification, in parallel, or when necessary, such as when a call is made. It may be a program to be processed.

  Further, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present technology.

  For example, the present technology can have a cloud computing configuration in which one function is shared and processed by a plurality of devices via a network.

  Further, each step described in the above-described flowchart can be executed by one device or in a shared manner by a plurality of devices.

  Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step can be executed by being shared by a plurality of devices in addition to being executed by one device.

  Further, the effects described in the present specification are merely examples and are not limited, and other effects may be present.

  Furthermore, the present technology can also be configured as follows.

(1)
An acquisition unit for acquiring encoded audio signals of a plurality of channels or a plurality of objects, and priority information of each of the audio signals at a predetermined time;
An audio signal decoding unit that decodes the encoded audio signal of a predetermined number of channels or objects according to the priority information based on the priority information.
(2)
The decoding apparatus according to (1), wherein the audio signal decoding unit decodes the encoded audio signal whose priority degree indicated by the priority information is equal to or higher than a predetermined degree.
(3)
The decoding device according to (2), wherein the acquisition unit changes the predetermined degree based on the priority information of audio signals of the plurality of channels or the plurality of objects at the predetermined time.
(4)
The acquisition unit acquires a plurality of the priority information for each of the audio signals,
The audio signal decoding unit decodes the encoded audio signal based on one piece of priority information selected from among the plurality of pieces of priority information (1) to (3). The decoding device according to one item.
(5)
The decoding device according to (4), wherein the plurality of pieces of priority information are generated for each of the computing capabilities in accordance with the computing capabilities of the decoding side of the encoded audio signal.
(6)
The decoding device according to any one of (1) to (5), further including: a priority information generation unit that generates the priority information based on the encoded audio signal.
(7)
The decoding apparatus according to (6), wherein the priority information generation unit generates the priority information based on a sound pressure or a spectral shape of an audio signal obtained from the encoded audio signal.
(8)
The audio signal decoding unit is configured to determine the predetermined time based on the priority information of the predetermined time and the priority information of a time before or after the predetermined time for each channel or object. The decoding apparatus according to (1), selects whether to decode the encoded audio signal.
(9)
When the decoding is performed, the signal obtained by the decoding is used as an output signal, and when the decoding is not performed, the zero data is used as an output signal for each channel or object for the output for the predetermined time. An addition unit that adds a signal and the output signal at a time before or after the predetermined time to generate an audio signal at the predetermined time;
The gain adjustment of the audio signal of the predetermined time is performed based on the priority information of the predetermined time and the priority information of a time before or after the predetermined time for each channel or object. The decoding device according to (1), further comprising: a gain adjustment unit to perform.
(10)
Gain-adjusting a high frequency power value based on the priority information of the predetermined time and the priority information of a time before or after the predetermined time for each channel or object The decoding apparatus according to (9), further comprising: a high band generation unit that generates a high band component of the audio signal of the predetermined time based on the gain-adjusted power value and the audio signal of the predetermined time. .
(11)
The device further comprises a high band generation unit that generates an audio signal of the predetermined time including the high band component based on the high band power value and the audio signal of the predetermined time for each channel or object.
The decoding apparatus according to (9), wherein the gain adjustment unit performs gain adjustment of the audio signal of the predetermined time in which a high frequency component is included.
(12)
The image processing apparatus further comprises a rendering unit that assigns audio signals of an object to a plurality of respective channels with predetermined gain values based on the priority information of the predetermined time, and generates audio signals of the plurality of channels (1) Decoding device described in.
(13)
Obtaining encoded audio signals of a plurality of channels or a plurality of objects, and priority information of each said audio signal at a predetermined time,
Decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information based on the priority information.
(14)
Obtaining encoded audio signals of a plurality of channels or a plurality of objects, and priority information of each said audio signal at a predetermined time,
A program that causes a computer to execute a process including the step of decoding the encoded audio signal of a predetermined number of channels or objects according to the priority information based on the priority information.
(15)
A priority information generator for generating priority information at predetermined times of audio signals of a plurality of channels or a plurality of objects;
An encoder for storing the priority information in a bit stream.
(16)
The encoding apparatus according to (15), wherein the priority information generation unit generates a plurality of the priority information for each of the audio signals.
(17)
The encoding device according to (16), wherein the priority information generation unit generates the priority information for each of the computing capabilities according to the computing capability of the decoding side of the encoded audio signal.
(18)
The encoding apparatus according to any one of (15) to (17), wherein the priority information generation unit generates the priority information based on a sound pressure or a spectral shape of the audio signal.
(19)
And a coding unit that codes audio signals of the plurality of channels or the plurality of objects,
The encoding apparatus according to any one of (15) to (18), wherein the packing unit stores the priority information and the encoded audio signal in the bit stream.
(20)
Generating priority information at predetermined times of audio signals of multiple channels or multiple objects,
Storing the priority information in a bitstream.
(21)
Generating priority information at predetermined times of audio signals of multiple channels or multiple objects,
A program that causes a computer to execute processing including the step of storing the priority information in a bitstream.

  11 coding device, 21 channel audio coding unit, 22 object audio coding unit, 23 metadata input unit, 24 packing unit, 51 coding unit, 52 priority information generation unit, 61 MDCT unit, 91 coding unit, 92 priority information generation unit, 101 MDCT unit, 151 decoding device, 161 unpacking / decoding unit, 162 rendering unit, 163 mixing unit, 191 priority information acquisition unit, 193 channel audio signal decoding unit, 194 output selection unit, 196 IMDCT unit, 198 object audio signal decoding unit, 199 output selection unit, 201 IMDCT unit, 231 priority information generation unit, 232 priority information generation unit, 271 overlap addition unit, 272 gain adjustment unit, 273 SBR processing unit, 274 Oh Rappu processing unit, 275 a gain adjustment unit, 276 SBR processor

Claims (3)

An acquiring unit for acquiring encoded audio signals of a plurality of objects from the supplied bitstream and priority information of the encoded audio signals of each of the objects at a predetermined time;
Based on the priority information, the encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority degree indicated in the priority information is the predetermined degree. An audio signal decoding unit that does not decode the encoded audio signal of the object that is less than a degree. The decryption device
Obtaining encoded audio signals of a plurality of objects from the supplied bitstream and priority information at predetermined times of the encoded audio signals of each said object,
Based on the priority information, the encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority degree indicated in the priority information is the predetermined degree. The encoded audio signal of the object, which is less than a degree, is not decoded. Obtaining encoded audio signals of a plurality of objects from the supplied bitstream and priority information at predetermined times of the encoded audio signals of each said object,
Based on the priority information, the encoded audio signal of the object whose priority degree indicated in the priority information is equal to or higher than a predetermined degree is decoded, and the priority degree indicated in the priority information is the predetermined degree. A program that causes a computer to execute a process including the step of not decoding the encoded audio signal of the object that is less than a degree.

Images