US20230260523A1 - Transmission device, transmission method, reception device and reception method - Google Patents
Transmission device, transmission method, reception device and reception method Download PDFInfo
- Publication number
- US20230260523A1 US20230260523A1 US18/307,605 US202318307605A US2023260523A1 US 20230260523 A1 US20230260523 A1 US 20230260523A1 US 202318307605 A US202318307605 A US 202318307605A US 2023260523 A1 US2023260523 A1 US 2023260523A1
- Authority
- US
- United States
- Prior art keywords
- encoded data
- stream
- audio
- group
- container
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims description 84
- 238000000034 method Methods 0.000 title claims description 13
- 238000012545 processing Methods 0.000 claims abstract description 80
- 238000003780 insertion Methods 0.000 claims description 18
- 230000037431 insertion Effects 0.000 claims description 18
- 238000004148 unit process Methods 0.000 claims description 2
- 229920003208 poly(ethylene sulfide) Polymers 0.000 description 18
- 229920006393 polyether sulfone Polymers 0.000 description 18
- 238000010586 diagram Methods 0.000 description 15
- 239000000872 buffer Substances 0.000 description 14
- 238000009877 rendering Methods 0.000 description 8
- 101150109471 PID2 gene Proteins 0.000 description 6
- 239000000284 extract Substances 0.000 description 5
- 238000013507 mapping Methods 0.000 description 5
- 101100190466 Caenorhabditis elegans pid-3 gene Proteins 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 101000609957 Homo sapiens PTB-containing, cubilin and LRP1-interacting protein Proteins 0.000 description 2
- 102100039157 PTB-containing, cubilin and LRP1-interacting protein Human genes 0.000 description 2
- 230000003321 amplification Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005401 electroluminescence Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 101100041819 Arabidopsis thaliana SCE1 gene Proteins 0.000 description 1
- 101100126625 Caenorhabditis elegans itr-1 gene Proteins 0.000 description 1
- 101100041822 Schizosaccharomyces pombe (strain 972 / ATCC 24843) sce3 gene Proteins 0.000 description 1
- 230000002730 additional effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/167—Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Time-Division Multiplex Systems (AREA)
- Reverberation, Karaoke And Other Acoustics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Communication Control (AREA)
- Television Systems (AREA)
Abstract
A processing load of a reception side is reduced when a plurality of types of audio data is transmitted. A predetermined format container is transmitted having a predetermined number of audio streams including a plurality of group encoded data. For example, the plurality of group encoded data includes either or both of channel encoded data and object encoded data. Attribute information indicating an attribute of each of the plurality of group encoded data is inserted into a layer of the container. For example, stream correspondence information indicating an audio stream including each of the plurality of group encoded data is further inserted into the layer of the container.
Description
- The present technology relates to a transmission device, a transmission method, a reception device, and a reception method, and in particular relates to a transmission device and the like for transmitting a plurality of types of audio data.
- Conventionally, as a stereoscopic (3D) acoustic technology, a technology has been devised for performing rendering by mapping encoded sample data to a speaker existing at an arbitrary position on the basis of metadata, (for example, see Patent Document 1).
-
- Patent Document 1: Japanese Patent Application National Publication (Laid-Open) No. 2014-520491
- It can be considered that object encoded data consisting of the encoded sample data and metadata is transmitted together with channel encoded data of 5.1 channels, 7.1 channels, and the like, and acoustic reproduction with enhanced realistic feeling can be achieved at a reception side.
- An object of the present technology is to reduce a processing load of the reception side when a plurality of types of audio data is transmitted.
- A concept of the present technology lies in
- a transmission device including:
- a transmission unit for transmitting a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data; and
- an information insertion unit for inserting attribute information indicating an attribute of each of the plurality of group encoded data, into a layer of the container.
- In the present technology, the predetermined format container having the predetermined number of audio streams including the plurality of group encoded data is transmitted by the transmission unit. For example, the plurality of group encoded data may include either or both of channel encoded data and object encoded data.
- The attribute information indicating the attribute of each of the plurality of group encoded data is inserted into the layer of the container by the information insertion unit. For example, the container may be a transport stream (MPEG-2 TS) adopted in a digital broadcasting standard. In addition, for example, the container may be a container of MP4 used in internet delivery and the like, or of another format.
- As described above, in the present technology, the attribute information indicating the attribute of each of the plurality of group encoded data included in the predetermined number of audio streams is inserted into the layer of the container. For that reason, at the reception side, the attribute of each of the plurality of group encoded data can be easily recognized before decoding of the encoded data, and only the necessary group encoded data can be selectively decoded to be used, and the processing load can be reduced.
- Incidentally, in the present technology, for example, the information insertion unit may further insert stream correspondence information indicating an audio stream including each of the plurality of group encoded data, into the layer of the container. In this case, for example, the container may be an MPEG2-TS, and the information insertion unit may insert the attribute information and the stream correspondence information into an audio elementary stream loop corresponding to any one audio stream of the predetermined number of audio streams existing under a program map table. As described above, the stream correspondence information is inserted into the layer of the container, whereby the audio stream including the necessary group encoded data can be easily recognized, and the processing load can be reduced at the reception side.
- For example, the stream correspondence information may be information indicating a correspondence between a group identifier for identifying each of the plurality of group encoded data and a stream identifier for identifying a stream of each of the predetermined number of audio streams. In this case, for example, the information insertion unit may further insert stream identifier information indicating a stream identifier of each of the predetermined number of audio streams, into the layer of the container. For example, the container may be an MPEG2-TS, and the information insertion unit may insert the stream identifier information into an audio elementary stream loop corresponding to each of the predetermined number of audio streams existing under the program map table.
- In addition, for example, the stream correspondence information may be information indicating a correspondence between the group identifier for identifying each of the plurality of group encoded data and a packet identifier to be attached during packetizing of each of the predetermined number of audio streams. In addition, for example, the stream correspondence information may be information indicating a correspondence between the group identifier for identifying each of the plurality of group encoded data and type information indicating a stream type of each of the predetermined number of audio streams.
- In addition, another concept of the present technology lies in
- a reception device including:
- a reception unit for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container; and
- a processing unit for processing the predetermined number of audio streams included in the container received, on the basis of the attribute information.
- In the present technology, the predetermined format container having the predetermined number of audio streams including the plurality of group encoded data is received by the reception unit. For example, the plurality of group encoded data may include either or both of channel encoded data and object encoded data. The attribute information indicating the attribute of each of the plurality of group encoded data is inserted into the layer of the container. The predetermined number of audio streams included in the container received is processed on the basis of the attribute information, by the processing unit.
- As described above, in the present technology, processing is performed of the predetermined number of audio streams included in the container received on the basis of the attribute information indicating the attribute of each of the plurality of group encoded data inserted into the layer of the container. For that reason, only the necessary group encoded data can be selectively decoded to be used, and the processing load can be reduced.
- Incidentally, in the present technology, for example, stream correspondence information indicating an audio stream including each of the plurality of group encoded data may be further inserted into the layer of the container, and the processing unit may process the predetermined number of audio streams on the basis of the stream correspondence information besides the attribute information. In this case, the audio stream including the necessary group encoded data can be easily recognized, and the processing load can be reduced.
- In addition, in the present technology, for example, the processing unit may selectively perform decoding processing to an audio stream including group encoded data holding an attribute conforming to a speaker configuration and user selection information, on the basis of the attribute information and the stream correspondence information.
- In addition, yet another concept of the present technology lies in
- a reception device including:
- a reception unit for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container;
- a processing unit for selectively acquiring predetermined group encoded data on the basis of the attribute information from the predetermined number of audio streams included in the container received, and reconfiguring an audio stream including the predetermined group encoded data; and
- a stream transmission unit for transmitting the audio stream reconfigured in the processing unit to an external device.
- In the present technology, the predetermined format container having the predetermined number of audio streams including the plurality of group encoded data is received by the reception unit. The attribute information indicating the attribute of each of the plurality of group encoded data is inserted into the layer of the container. The predetermined group encoded data is selectively acquired on the basis of the attribute information from the predetermined number of audio streams, by the processing unit, and the audio stream including the predetermined group encoded data is reconfigured. Then, the audio stream reconfigured is transmitted to the external device, by the stream transmission unit.
- As described above, in the present technology, on the basis of the attribute information indicating the attribute of each of the plurality of group encoded data inserted into the layer of the container, the predetermined group encoded data is selectively acquired from the predetermined number of audio streams, and the audio stream to be transmitted to the external device is reconfigured. The necessary group encoded data can be easily acquired, and the processing load can be reduced.
- Incidentally, in the present technology, for example, stream correspondence information indicating an audio stream including each of the plurality of group encoded data may be further inserted into the layer of the container, and the processing unit may selectively acquire the predetermined group encoded data from the predetermined number of audio streams on the basis of the stream correspondence information, besides the attribute information. In this case, the audio stream including the predetermined group encoded data can be easily recognized, and the processing load can be reduced.
- According to the present technology, the processing load of the reception side can be reduced when the plurality of types of audio data is transmitted. Incidentally, the advantageous effects described in this specification are merely examples, and the advantageous effects of the present technology are not limited to them and may include additional effects.
-
FIG. 1 is a block diagram showing an example configuration of a transmission/reception system as an embodiment. -
FIG. 2 is a diagram showing a structure of an audio frame (1024 samples) in 3D audio transmission data. -
FIG. 3 is a diagram showing an example configuration of the 3D audio transmission data. -
FIGS. 4(a) and 4(b) are diagrams schematically showing example configurations of the audio frame when transmission of the 3D audio transmission data is performed in one stream, and when the transmission is performed in multiple streams, respectively. -
FIG. 5 is a diagram showing a group division example when the transmission is performed in three streams in the example configuration of the 3D audio transmission data. -
FIG. 6 is a diagram showing a correspondence between a group and a sub stream in the group division example (three divisions), and the like. -
FIG. 7 is a diagram showing a group division example in which the transmission is performed in two streams in the example configuration of the 3D audio transmission data. -
FIG. 8 is a diagram showing a correspondence between a group and a sub stream in the group division example (two divisions), and the like. -
FIG. 9 is a block diagram showing an example configuration of a stream generation unit included in a service transmitter. -
FIG. 10 is a diagram showing a structural example of a 3D audio stream configuration descriptor. -
FIG. 11 is a diagram showing details of main information in the structural example of the 3D audio stream configuration descriptor. -
FIGS. 12(a) and 12(b) are diagrams respectively showing a structural example of a 3D audio sub stream ID descriptor, and a detail of main information in the structural example. -
FIG. 13 is a diagram showing an example configuration of a transport stream. -
FIG. 14 is a block diagram showing an example configuration of a service receiver. -
FIG. 15 is a flowchart showing an example of audio decoding control processing of a CPU in the service receiver. -
FIG. 16 is a block diagram showing another example configuration of the service receiver. - The following is a description of a mode for carrying out the invention (the mode will be hereinafter referred to as the “embodiment”). Incidentally, explanation will be made in the following order.
- 1. Embodiment
- 2. Modification
- [Example Configuration of a Transmission/Reception System]
-
FIG. 1 shows an example configuration of a transmission/reception system 10 as an embodiment. The transmission/reception system 10 is configured by aservice transmitter 100 and aservice receiver 200. Theservice transmitter 100 transmits a transport stream TS loaded on a broadcast wave or a network packet. The transport stream TS has a video stream, and a predetermined number of audio streams including a plurality of group encoded data. -
FIG. 2 shows a structure of an audio frame (1024 samples) in 3D audio transmission data dealt with in the embodiment. The audio frame consists of multiple MPEG audio stream packets (mpeg Audio Stream Packets). Each of the MPEG audio stream packets is configured by a header (Header) and a payload (Payload). - The header holds information, such as a packet type (Packet Type), a packet label (Packet Label), and a packet length (Packet Length). Information defined by the packet type of the header is disposed in the payload. In the payload information, there exist “SYNC” information corresponding to a synchronization start code, “Frame” information being actual data of the 3D audio transmission data, and “Config” information indicating a configuration of the “Frame” information.
- The “Frame” information includes object encoded data and channel encoded data configuring the 3D audio transmission data. Here, the channel encoded data is configured by encoded sample data such as a Single Channel Element (SCE), a Channel Pair Element (CPE), and a Low Frequency Element (LFE). In addition, the object encoded data is configured by the encoded sample data of the Single Channel Element (SCE), and metadata for performing rendering by mapping the encoded sample data to a speaker existing at an arbitrary position. The metadata is included as an extension element (Ext element).
-
FIG. 3 shows an example configuration of the 3D audio transmission data. This example consists of one channel encoded data and two object encoded data. The one channel encoded data is channel encoded data (CD) of 5.1 channels, and consists of encoded sample data of SCE1, CPE1.1, CPE1.2, LFE1. - The two object encoded data are immersive audio object (Immersive audio object: IAO) encoded data and speech dialog object (Speech Dialog object: SDO) encoded data. The immersive audio object encoded data is object encoded data for an immersive sound, and consists of encoded sample data SCE2, and metadata EXE_El (Object metadata) 2 for performing rendering by mapping the encoded sample data to the speaker existing at the arbitrary position.
- The speech dialog object encoded data is object encoded data for a speech language. In this example, speech dialog object encoded data exist respectively corresponding to
language 1 andlanguage 2. The speech dialog object encoded data corresponding to thelanguage 1 consists of encoded sample data SCE3, and metadata EXE_El (Object metadata) 3 for performing rendering by mapping the encoded sample data to the speaker existing at the arbitrary position. In addition, the speech dialog object encoded data corresponding to thelanguage 2 consists of encoded sample data SCE4, and metadata EXE_El (Object metadata) 4 for performing rendering by mapping the encoded sample data to the speaker existing at the arbitrary position. - The encoded data is distinguished by a concept of a group (Group) by type. In the example shown, the encoded channel data of 5.1 channels is in a
group 1, the immersive audio object encoded data is in agroup 2, the speech dialog object encoded data of thelanguage 1 is in agroup 3, and the speech dialog object encoded data of thelanguage 2 is in agroup 4. - In addition, the data that can be selected between the groups at a reception side is registered with a switch group (SW Group) and encoded. In addition, the groups can be bundled into a preset group (preset Group), and can be reproduced according to a use case. In the example shown, the
group 1, thegroup 2, and thegroup 3 are bundled into apreset group 1, and thegroup 1, thegroup 2, and thegroup 4 are bundled into apreset group 2. - Returning to
FIG. 1 , theservice transmitter 100 transmits the 3D audio transmission data including the plurality of group encoded data in one stream, or multiple streams (Multiple stream), as described above. -
FIG. 4 (a) schematically shows an example configuration of the audio frame when transmission is performed in one stream in the example configuration of the 3D audio transmission data ofFIG. 3 . In this case, the one stream includes the channel encoded data (CD), the immersive audio object encoded data (IAO), and the speech dialog object encoded data (SDO), together with the “SYNC” information and the “Config” information. -
FIG. 4 (b) schematically shows an example configuration of the audio frame when the transmission is performed in multiple streams (each of the streams is referred to as “sub stream,” if appropriate), here three streams, in the example configuration of the 3D audio transmission data ofFIG. 3 . In this case, asub stream 1 includes the channel encoded data (CD), together with the “SYNC” information and the “Config” information. In addition, asub stream 2 includes the immersive audio object encoded data (IAO), together with the “SYNC” information and the “Config” information. Further, asub stream 3 includes the speech dialog object encoded data (SDO), together with the “SYNC” information and the “Config” information. -
FIG. 5 shows a group division example when the transmission is performed in three streams in the example configuration of the 3D audio transmission data ofFIG. 3 . In this case, thesub stream 1 includes the channel encoded data (CD) distinguished as thegroup 1. In addition, thesub stream 2 includes the immersive audio object encoded data (IAO) distinguished as thegroup 2. In addition, thesub stream 3 includes the speech dialog object encoded data (SDO) of thelanguage 1 distinguished as thegroup 3, and the speech dialog object encoded data (SDO) of thelanguage 2 distinguished as thegroup 4. -
FIG. 6 shows a correspondence between a group and a sub stream in the group division example (three divisions) ofFIG. 5 , and the like. Here, a group ID (group ID) is an identifier for identifying the group. An attribute (attribute) indicates an attribute of each of the group encoded data. A switch group ID (switch Group ID) is an identifier for identifying the switching group. A preset group ID (preset Group ID) is an identifier for identifying the preset group. A sub stream ID (sub Stream ID) is an identifier for identifying the sub stream. - The shown correspondence indicates that the encoded data belonging to the
group 1 is the channel encoded data, does not configure the switch group, and is included in thesub stream 1. In addition, the shown correspondence indicates that the encoded data belonging to thegroup 2 is the object encoded data (immersive audio object encoded data) for the immersive sound, does not configure the switch group, and is included in thesub stream 2. - In addition, the shown correspondence indicates that the encoded data belonging to the
group 3 is the object encoded data (speech dialog object encoded data) for the speech language of thelanguage 1, configures theswitch group 1, and is included in thesub stream 3. In addition, the shown correspondence indicates that the encoded data belonging to thegroup 4 is the object encoded data (speech dialog object encoded data) for the speech language of thelanguage 2, configures theswitch group 1, and is included in thesub stream 3. - In addition, the shown correspondence indicates that the
preset group 1 includes thegroup 1, thegroup 2, and thegroup 3. Further, the shown correspondence indicates that thepreset group 2 includes thegroup 1, thegroup 2, and thegroup 4. -
FIG. 7 shows a group division example in which the transmission is performed in two streams in the example configuration of the 3D audio transmission data ofFIG. 3 . In this case, thesub stream 1 includes the channel encoded data (CD) distinguished as thegroup 1, and the immersive audio object encoded data (IAO) distinguished as thegroup 2. In addition, thesub stream 2 includes the speech dialog object encoded data (SDO) of thelanguage 1 distinguished as thegroup 3, and the speech dialog object encoded data (SDO) of thelanguage 2 distinguished as thegroup 4. -
FIG. 8 shows a correspondence between a group and a sub stream in the group division example (two divisions) ofFIG. 7 , and the like. The shown correspondence indicates that the encoded data belonging to thegroup 1 is the channel encoded data, does not configure the switch group, and is included in thesub stream 1. In addition, the shown correspondence indicates that the encoded data belonging to thegroup 2 is the object encoded data (immersive audio object encoded data) for the immersive sound, does not configure the switch group, and is included in thesub stream 1. - In addition, the shown correspondence indicates that the encoded data belonging to the
group 3 is the object encoded data (speech dialog object encoded data) for the speech language of thelanguage 1, configures theswitch group 1, and is included in thesub stream 2. In addition, the shown correspondence indicates that the encoded data belonging to thegroup 4 is the object encoded data (speech dialog object encoded data) for the speech language of thelanguage 2, configures theswitch group 1, and is included in thesub stream 2. - In addition, the shown correspondence indicates that the
preset group 1 includes thegroup 1, thegroup 2, and thegroup 3. Further, the shown correspondence indicates that thepreset group 2 includes thegroup 1, thegroup 2, and thegroup 4. - Returning to
FIG. 1 , theservice transmitter 100 inserts attribute information indicating an attribute of each of the plurality of group encoded data included in the 3D audio transmission data, into a layer of the container. In addition, theservice transmitter 100 inserts stream correspondence information indicating an audio stream including each of the plurality of group encoded data, into the layer of the container. In the embodiment, the stream correspondence information is, for example, information indicating a correspondence between a group ID and a stream identifier. - The
service transmitter 100 inserts these attribute information and stream correspondence information as a descriptor in, for example, any one audio stream of the predetermined number of audio streams existing under a program map table (Program Map Table: PMT), for example, an audio elementary stream loop corresponding to the most basic stream. - In addition, the
service transmitter 100 inserts stream identifier information indicating a stream identifier indicating a stream identifier of each of the predetermined number of audio streams, into the layer of the container. Theservice transmitter 100 inserts the stream identifier information as a descriptor into an audio elementary stream loop corresponding to each of the predetermined number of audio streams existing under the program map table (Program Map Table: PMT), for example. - The
service receiver 200 receives the transport stream TS loaded on the broadcast wave or the network packet and transmitted from theservice transmitter 100. The transport stream TS has the predetermined number of audio streams including the plurality of group encoded data configuring the 3D audio transmission data, besides the video stream, as described above. Then, into the layer of the container, the attribute information indicating the attribute of each of the plurality of group encoded data included in the 3D audio transmission data is inserted, and the stream correspondence information indicating the audio stream including each of the plurality of group encoded data is inserted. - The
service receiver 200 selectively performs decoding processing to an audio stream including group encoded data holding an attribute conforming to a speaker configuration and user selection information, on the basis of the attribute information and the stream correspondence information, and obtains an audio output of the 3D audio. - [Stream Generation Unit of Service Transmitter]
-
FIG. 9 shows an example configuration of astream generation unit 110 included in theservice transmitter 100. Thestream generation unit 110 has avideo encoder 112, anaudio encoder 113, and amultiplexer 114. Here, an example is assumed in which audio transmission data consists of one encoded channel data and two object encoded data as shown inFIG. 3 . - The
video encoder 112 inputs video data SV, and performs encoding to the video data SV to generate a video stream (video elementary stream). Theaudio encoder 113 inputs the channel data and the immersive audio and speech dialog object data, as audio data SA. - The
audio encoder 113 performs encoding to the audio data SA, and obtains the 3D audio transmission data. The 3D audio transmission data includes the channel encoded data (CD), the immersive audio object encoded data (IAO), and the speech dialog object encoded data (SDO), as shown inFIG. 3 . Then, theaudio encoder 113 generates one or multiple audio streams (audio elementary streams) including the plurality of, here four, group encoded data (seeFIGS. 4(a), 4(b) ). - The
multiplexer 114 packetizes each of the video stream output from thevideo encoder 112 and the predetermined number of audio streams output from theaudio encoder 113, into a PES packet, and further into a transport packet to multiplex the streams, and obtains the transport stream TS as a multiplexed stream. - In addition, the
multiplexer 114 inserts the attribute information indicating the attribute of each of the plurality of group encoded data, and the stream correspondence information indicating the audio stream including each of the plurality of group encoded data, under the program map table (PMT). Themultiplexer 114 inserts these pieces of information into, for example, an audio elementary stream loop corresponding to the most basic stream, by using a 3D audio stream configuration descriptor (3Daudio_stream_config_descriptor). The descriptor will be described later in detail. - In addition, the
multiplexer 114 inserts the stream identifier information indicating the stream identifier of each of the predetermined number of audio streams, under the program map table (PMT). Themultiplexer 114 inserts the information into an audio elementary stream loop corresponding to each of the predetermined number of audio streams, by using a 3D audio sub stream ID descriptor (3Daudio_substreamID_descriptor). The descriptor will be described later in detail. - Operation of the
stream generation unit 110 shown inFIG. 9 is now briefly described. The video data is supplied to thevideo encoder 112. In thevideo encoder 112, encoding is performed to the video data SV, and a video stream including encoded video data is generated. The video stream is supplied to themultiplexer 114. - The audio data SA is supplied to the
audio encoder 113. The audio data SA includes the channel data, and the immersive audio and speech dialog object data. In theaudio encoder 113, encoding is performed to the audio data SA, and the 3D audio transmission data is obtained. - The 3D audio transmission data includes the immersive audio object encoded data (IAO), and the speech dialog object encoded data (SDO), besides the channel encoded data (CD) (see
FIG. 3 ). Then, in theaudio encoder 113, the one or multiple audio streams including four group encoded data are generated (seeFIGS. 4 (a), 4(b) ). - The video stream generated by the
video encoder 112 is supplied to themultiplexer 114. In addition, the audio stream generated by theaudio encoder 113 is supplied to themultiplexer 114. In themultiplexer 114, the stream supplied from each encoder is packetized into the PES packet, and further into the transport packet to be multiplexed, and the transport stream TS is obtained as the multiplexed stream. - In addition, in the
multiplexer 114, the 3D audio stream configuration descriptor is inserted into, for example, the audio elementary stream loop corresponding to the most basic stream. The descriptor includes the attribute information indicating the attribute of each of the plurality of group encoded data, and the stream correspondence information indicating the audio stream including each of the plurality of group encoded data. - In addition, in the
multiplexer 114, the 3D audio sub stream ID descriptor is inserted into the audio elementary stream loop corresponding to each of the predetermined number of audio streams. The descriptor includes the stream identifier information indicating the stream identifier of each of the predetermined number of audio streams. - [Details of 3D audio Stream Configuration Descriptor]
-
FIG. 10 shows a structural example (Syntax) of the 3D audio stream configuration descriptor (3Daudio_stream_config_descriptor). In addition,FIG. 11 shows details of main information (Semantics) in the structural example. - An eight bit field of a “descriptor_tag” indicates a descriptor type. Here, it is indicated that the descriptor is the 3D audio stream configuration descriptor. An eight bit field of a “descriptor_length” indicates a length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.
- An eight bit field of a “NumOfGroups, N” indicates the number of groups. An eight bit field of a “NumOfPresetGroups, P” indicates the number of preset groups. An eight bit field of a “groupID,” an eight bit field of an “attribute_of_groupID,” an eight bit field of a “SwitchGroupID,” and an eight bit field of an “audio_substreamID” are repeated by the number of groups.
- The field of the “groupID” indicates a group identifier. The field of the “attribute_of_groupID” indicates an attribute of the group encoded data. The field of the “SwitchGroupID” is an identifier indicating a switch group to which the group belongs. “0” indicates that the group does not belong to any switch group. Other than “0” indicates a switch group to be caused to belong. The “audio_substreamID” is an identifier indicating an audio sub stream including the group.
- In addition, an eight bit field of a “presetGroupID” and an eight bit field of a “NumOfGroups_in_preset, R” are repeated by the number of preset groups. The field of the “presetGroupID” is an identifier indicating a bundle presetting a group. The field of the “NumOfGroups_in_preset, R” indicates the number of groups belonging to the preset group. Then, for each preset group, the eight bit field of the “groupID” is repeated by the number of groups belonging to the preset group, and the group belonging to the preset group is indicated. The descriptor may be disposed under an extended descriptor.
- [Details of 3D audio Sub stream ID Descriptor]
FIG. 12 (a) shows a structural example (Syntax) of a 3D audio sub stream ID descriptor (3Daudio_substreamID_descriptor). In addition,FIG. 12 (b) shows a detail of main information (Semantics) in the structural example. - An eight bit field of a “descriptor_tag” indicates a descriptor type. Here, it is indicated that the descriptor is the 3D audio sub stream ID descriptor. An eight bit field of a “descriptor_length” indicates a length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor. An eight bit field of an “audio_substreamID” indicates an audio sub stream identifier. The descriptor may be disposed under an extended descriptor.
- [Configuration of Transport Stream TS]
-
FIG. 13 shows an example configuration of a transport stream TS. The example configuration corresponds to a case in which transmission is performed in two streams of the 3D audio transmission data (seeFIG. 7 ). In the example configuration, there exists a video stream PES packet “video PES” identified by PID1. In addition, in the example configuration, there exist two audio stream (audio sub stream) PES packets “audio PESs” respectively identified by PID2, PID3. The PES packet consists of a PES header (PES_header) and a PES payload (PES_payload). In the PES header, time stamps of DTS, PTS are inserted. The time stamps of the PID2 and PID3 are appropriately attached such that the time stamps are matched each other during multiplexing, whereby synchronization between them can be ensured for the entire system. - Here, the audio stream PES packet “audio PES” identified by the PID2 includes the channel encoded data (CD) distinguished as the
group 1 and the immersive audio object encoded data (IAO) distinguished as thegroup 2. In addition, the audio stream PES packet “audio PES” identified by the PID3 includes the speech dialog object encoded data (SDO) of thelanguage 1 distinguished as thegroup 3 and the speech dialog object encoded data (SDO) of thelanguage 2 distinguished as thegroup 4. - In addition, the transport stream TS includes the Program Map Table (PMT) as Program Specific Information (PSI). The PSI is information indicating a program to which each elementary stream included in the transport stream belongs. In the PMT, a program loop (Program loop) exists describing information related to the entire program.
- In addition, in the PMT, an elementary stream loop exists holding information related to each elementary stream. In the example configuration, a video elementary stream loop (video ES loop) exists corresponding to the video stream, and audio elementary stream loops (audio ES loops) exist respectively corresponding to two audio streams.
- In the video elementary stream loop (video ES loop), information is disposed such as a stream type, and a PID (packet identifier) corresponding to the video stream, and a descriptor is also disposed describing information related to the video stream. A value of a “Stream type” of the video stream is set to “0x24,” and the PID information indicates the PID1 given to the video stream PES packet “video PES” as described above. A HEVC descriptor is disposed as one of the descriptors.
- In addition, in the audio elementary stream loop (audio ES loop), the information is disposed such as a stream type, and a PID (packet identifier) corresponding to the audio stream, and a descriptor is also disposed describing information related to the audio stream. A value of a “Stream_type” of the audio stream is set to “0x2C,” and the PID information indicates the PID2 given to the audio stream PES packet “audio PES” as described above.
- In the audio elementary stream loop (audio ES loop) corresponding to the audio stream identified by the PID2, both of the above-described 3D audio stream configuration descriptor and the 3D audio sub stream ID descriptor are disposed. In addition, in the audio elementary stream loop (audio ES loop) corresponding to the audio stream identified by the PID2, only the above described 3D audio sub stream ID descriptor is disposed.
- [Example Configuration of Service Receiver]
-
FIG. 14 shows an example configuration of theservice receiver 200. Theservice receiver 200 has areception unit 201, ademultiplexer 202, avideo decoder 203, avideo processing circuit 204, apanel drive circuit 205, and adisplay panel 206. In addition, theservice receiver 200 has multiplexing buffers 211-1 to 211-N, acombiner 212, a3D audio decoder 213, an audiooutput processing circuit 214, and aspeaker system 215. In addition, theservice receiver 200 has aCPU 221, aflash ROM 222, aDRAM 223, aninternal bus 224, a remotecontrol reception unit 225, and aremote control transmitter 226. - The
CPU 221 controls operation of each unit of theservice receiver 200. Theflash ROM 222 stores control software and keeps data. TheDRAM 223 configures a work area of theCPU 221. TheCPU 221 deploys the software and data read from theflash ROM 222 on theDRAM 223 and activates the software to control each unit of theservice receiver 200. - The remote
control reception unit 225 receives a remote control signal (remote control code) transmitted from theremote control transmitter 226, and supplies the signal to theCPU 221. TheCPU 221 controls each unit of theservice receiver 200 on the basis of the remote control code. TheCPU 221, theflash ROM 222, and theDRAM 223 are connected to theinternal bus 224. - The
reception unit 201 receives the transport stream TS loaded on the broadcast wave or the network packet and transmitted from theservice transmitter 100. The transport stream TS has the predetermined number of audio streams including the plurality of group encoded data configuring the 3D audio transmission data, besides the video stream. - The
demultiplexer 202 extracts a video stream packet from the transport stream TS and transmits the packet to thevideo decoder 203. Thevideo decoder 203 reconfigures the video stream from the video packet extracted by thedemultiplexer 202, and performs decoding processing to obtain uncompressed video data. - The
video processing circuit 204 performs scaling processing, image quality adjustment processing, and the like to the video data obtained by thevideo decoder 203, and obtains video data for display. Thepanel drive circuit 205 drives thedisplay panel 206 on the basis of image data for display obtained by thevideo processing circuit 204. Thedisplay panel 206 is configured by, for example, a Liquid Crystal Display (LCD), an organic electroluminescence (EL) display. - In addition, the
demultiplexer 202 extracts information such as various descriptors from the transport stream TS, and transmits the information to theCPU 221. The various descriptors include the above-described 3D audio stream configuration descriptor (3Daudio_stream_config_descriptor) and 3D audio sub stream ID descriptor (3Daudio_substreamID_descriptor) (seeFIG. 13 ). - The
CPU 221 recognizes an audio stream including the group encoded data holding the attribute conforming to the speaker configuration and viewer (user) selection information, on the basis of the attribute information indicating the attribute of each of the group encoded data, stream relationship information indicating the audio stream (sub stream) including each group, and the like included in these descriptors. - In addition, the
demultiplexer 202 selectively extracts by a PID filter one or multiple audio stream packets including the group encoded data holding the attribute conforming to the speaker configuration and viewer (user) selection information, of the predetermined number of audio streams included in the transport stream TS, under the control of theCPU 221. - The multiplexing buffers 211-1 to 211-N respectively take in the audio streams extracted by the
demultiplexer 202. Here, the number N of multiplexing buffers 211-1 to 211-N is a necessary and sufficient number, and the number of audio streams extracted by thedemultiplexer 202 is used, in actual operation. - The
combiner 212 reads the audio stream for each audio frame from each of the multiplexing buffers respectively taking in the audio streams extracted by thedemultiplexer 202, of the multiplexing buffers 211-1 to 211-N, and supplies the audio stream to the3D audio decoder 213 as the group encoded data holding the attribute conforming to the speaker configuration and viewer (user) selection information. - The
3D audio decoder 213 performs decoding processing to the encoded data supplied from thecombiner 212, and obtains audio data for driving each speaker of thespeaker system 215. Here, three cases can be considered, which are a case in which the encoded data to be subjected to the decoding processing includes only the channel encoded data, a case in which the encoded data includes only the object encoded data, and further a case in which the encoded data includes both of the channel encoded data and the object encoded data. - When decoding the channel encoded data, the
3D audio decoder 213 performs processing of downmix and upmix for the speaker configuration of thespeaker system 215, and obtains the audio data for driving each speaker. In addition, when decoding the object encoded data, the3D audio decoder 213 calculates speaker rendering (mixing ratio for each speaker) on the basis of the object information (metadata), and mixes object audio data with the audio data for driving each speaker according to the calculation result. - The audio
output processing circuit 214 performs necessary processing such as D/A conversion and amplification, to the audio data for driving each speaker obtained by the3D audio decoder 213, and supplies the audio data to thespeaker system 215. Thespeaker system 215 includes multiple speakers of multiple channels, for example 2 channels, 5.1 channels, 7.1 channels, and 22.2 channels. - Operation of the
service receiver 200 shown inFIG. 14 is now briefly described. In thereception unit 201, the transport stream TS is received loaded on the broadcast wave or the network packet and transmitted from theservice transmitter 100. The transport stream TS has the predetermined number of audio streams including the plurality of group encoded data configuring the 3D audio transmission data, besides the video stream. The transport stream TS is supplied to thedemultiplexer 202. - In the
demultiplexer 202, the video stream packet is extracted from the transport stream TS, and supplied to thevideo decoder 203. In thevideo decoder 203, the video stream is reconfigured from the video packet extracted by thedemultiplexer 202, and the decoding processing is performed, and the uncompressed video data is obtained. The video data is supplied to thevideo processing circuit 204. - In the
video processing circuit 204, the scaling processing, the image quality adjustment processing, and the like are performed to the video data obtained by thevideo decoder 203, and the video data for display is obtained. The video data for display is supplied to thepanel drive circuit 205. In thepanel drive circuit 205, thedisplay panel 206 is driven on the basis of the video data for display. Thus, an image is displayed corresponding to the video data for display, on thedisplay panel 206. - In addition, in the
demultiplexer 202, the information such as the various descriptors is extracted from the transport stream TS, and is transmitted to theCPU 221. The various descriptors include the 3D audio stream configuration descriptor and the 3D audio sub stream ID descriptor. In theCPU 221, the audio stream (sub stream) is recognized including the group encoded data holding the attribute conforming to the speaker configuration and viewer (user) selection information, on the basis of the attribute information, the stream relationship information, and the like included in these descriptors. - In addition, in the
demultiplexer 202, the one or multiple audio stream packets are selectively extracted by the PID filter, the audio stream packets including the group encoded data holding the attribute conforming to the speaker configuration and viewer selection information, of the predetermined number of audio streams included in the transport stream TS, under the control of theCPU 221. - The audio streams extracted by the
demultiplexer 202 are respectively taken in the corresponding multiplexing buffers of the multiplexing buffers 211-1 to 211-N. In thecombiner 212, the audio stream is read for each audio frame from each of the multiplexing buffers respectively taking in the audio streams, and is supplied to the3D audio decoder 213 as the group encoded data holding the attribute conforming to the speaker configuration and viewer selection information. - In the
3D audio decoder 213, the decoding processing is performed to the encoded data supplied from thecombiner 212, and the audio data is obtained for driving each speaker of thespeaker system 215. - Here, when the channel encoded data is decoded, the processing of downmix and upmix is performed for the speaker configuration of the
speaker system 215, and the audio data is obtained for driving each speaker. In addition, when the object encoded data is decoded, the speaker rendering (mixing ratio for each speaker) is calculated on the basis of the object information (metadata), and the object audio data is mixed with the audio data for driving each speaker according to the calculation result. - The audio data for driving each speaker obtained by the
3D audio decoder 213 is supplied to the audiooutput processing circuit 214. In the audiooutput processing circuit 214, the necessary processing such as the D/A conversion and amplification is performed to the audio data for driving each speaker. Then, the audio data after the processing is supplied to thespeaker system 215. Thus, an audio output is obtained corresponding to a display image on thedisplay panel 206 from thespeaker system 215. -
FIG. 15 shows an example of audio decoding control processing of theCPU 221 in theservice receiver 200 shown inFIG. 14 . TheCPU 221 starts the processing, in step ST1. Then, theCPU 221 detects a receiver speaker configuration, that is, the speaker configuration of thespeaker system 215, in step ST2. Next, theCPU 221 obtains selection information related to an audio output by a viewer (user), in step ST3. - Next, the
CPU 221 reads the “groupID,” “attribute_of_GroupID,” “switchGroupID,” “presetGroupID,” and “Audio_substreamID” of the 3D audio stream configuration descriptor (3Daudio_stream_config_descriptor), in step ST4. Then, theCPU 221 recognizes the sub stream ID (substreamID) of the audio stream (sub stream) to which the group holding the attribute conforming to the speaker configuration and viewer selection information belongs, in step ST5. - Next, the
CPU 221 collates the sub stream ID (substreamID) recognized with the sub stream ID (substreamID) of the 3D audio sub stream ID descriptor (3Daudio_substreamID_descriptor) of each audio stream (sub stream), and selects a matched one by the PID filter (PID filter), and takes the one in each of the multiplexing buffers, in step ST6. Then, theCPU 221 reads the audio stream (sub stream) for each audio frame from each of the multiplexing buffers, and supplies the necessary group encoded data to the3D audio decoder 213, in step ST7. - Next, the
CPU 221 determines whether or not to decode the object encoded data, in step ST8. When decoding the object encoded data, theCPU 221 calculates the speaker rendering (mixing ratio for each speaker) by azimuth (azimuth information) and elevation (elevation information) on the basis of the object information (metadata), in step ST9. After that, theCPU 221 proceeds to step ST10. Incidentally, when not decoding the object encoded data in step ST8, theCPU 221 immediately proceeds to step ST10. - The
CPU 221 determines whether or not to decode the channel encoded data, in step ST10. When decoding the channel encoded data, theCPU 221 performs the processing of downmix and upmix for the speaker configuration of thespeaker system 215, and obtains the audio data for driving each speaker, in step ST11. After that, theCPU 221 proceeds to step ST12. Incidentally, when not decoding the object encoded data in step ST10, theCPU 221 immediately proceeds to step ST12. - The
CPU 221 mixes the object audio data with the audio data for driving each speaker according to the calculation result in step ST9 when decoding the object encoded data, and then performs dynamic range control, in step ST12. After that, the CPU 21 ends the processing, in step ST13. Incidentally, when not decoding the object encoded data, theCPU 221 skips step ST12. - As described above, in the transmission/
reception system 10 shown inFIG. 1 , theservice transmitter 100 inserts the attribute information indicating the attribute of each of the plurality of group encoded data included in the predetermined number of audio streams, into the layer of the container. For that reason, at the reception side, the attribute of each of the plurality of group encoded data can be easily recognized before decoding of the encoded data, and only the necessary group encoded data can be selectively decoded to be used, and the processing load can be reduced. - In addition, in the transmission/
reception system 10 shown inFIG. 1 , theservice transmitter 100 inserts the stream correspondence information indicating the audio stream including each of the plurality of group encoded data, into the layer of the container. For that reason, at the reception side, the audio stream including the necessary group encoded data can be easily recognized, and the processing load can be reduced. - Incidentally, in the above-described embodiment, the
service receiver 200 is configured to selectively extract the audio stream including the group encoded data holding the attribute conforming to the speaker configuration and viewer selection information, from the multiple audio streams (sub streams) transmitted from theservice transmitter 100, and to perform the decoding processing to obtain the audio data for driving a predetermined number of speakers. - However, it can also be considered, as the service receiver, to selectively extract one or multiple audio streams holding the group encoded data holding the attribute conforming to the speaker configuration and viewer selection information, from the multiple audio streams (sub streams) transmitted from the
service transmitter 100, to reconfigure an audio stream holding the group encoded data holding the attribute conforming to the speaker configuration and viewer selection information, and to deliver the reconfigured audio stream to a device (including a DLNA device) connected to a local network. -
FIG. 16 shows an example configuration of aservice receiver 200A for delivering the reconfigured audio stream to the device connected to the local network as described above. InFIG. 16 , the components equivalent to components shown inFIG. 14 are denoted by the same reference numerals as those used inFIG. 14 , and detailed explanation of them is not repeated herein. - In the
demultiplexer 202, one or multiple audio stream packets are selectively extracted by the PID filter, the audio stream packets including the group encoded data holding the attribute conforming to the speaker configuration and viewer selection information, of the predetermined number of audio streams included in the transport stream TS, under the control of theCPU 221. - The audio streams extracted by the
demultiplexer 202 are respectively taken in the corresponding multiplexing buffers of the multiplexing buffers 211-1 to 211-N. In thecombiner 212, the audio stream is read for each audio frame from each of the multiplexing buffers respectively taking in the audio streams, and is supplied to astream reconfiguration unit 231. - In the
stream reconfiguration unit 231, the predetermined group encoded data is selectively acquired holding the attribute conforming to the speaker configuration and viewer selection information, and the audio stream is reconfigured holding the predetermined group encoded data. The reconfigured audio stream is supplied to adelivery interface 232. Then, the delivery (transmission) is performed from thedelivery interface 232 to adevice 300 connected to the local network. - The local network connection includes Ethernet connection, and wireless connection such as “WiFi” or “Bluetooth.” Incidentally, “WiFi” and “Bluetooth” are registered trademarks.
- In addition, the
device 300 includes a surround speaker, a second display, and an audio output device attached to a network terminal. Thedevice 300 receiving delivery of the reconfigured audio stream performs the decoding processing similar to that of the3D audio decoder 213 in theservice receiver 200 ofFIG. 14 , and obtains the audio data for driving the predetermined number of speakers. - In addition, as the service receiver, a configuration can also be considered in which the above-described reconfigured audio stream is transmitted to a device connected via a digital interface such as “High-Definition Multimedia Interface (HDMI),” “Mobile High definition Link (MHL)”, or “DisplayPort.” Incidentally, “HDMI” and “MHL” are registered trademarks.
- In addition, in the above-described embodiment, the stream correspondence information inserted into the layer of the container is the information indicating a correspondence between the group ID and the sub stream ID. That is, the sub stream ID is used for associating the group and the audio stream (sub stream) with each other. However, it can also be considered to use the packet identifier (Packet ID: PID) or the stream type (stream type) for associating the group and the audio stream (sub stream) with each other. Incidentally, when the stream type is used, it is necessary to change the stream type of each audio stream (sub stream).
- In addition, in the above-described embodiment, an example has been shown in which the attribute information of each of the group encoded data is transmitted by providing the field of the “attribute_of_groupID” (see
FIG. 10 ). However, the present technology includes a method in which the type (attribute) of the encoded data can be recognized when a specific group ID is recognized, by defining a special meaning for a value of the group ID (GroupID) itself between the transmitter and the receiver. In this case, the group ID functions as group identifier, and also functions as the attribute information of the group encoded data, so that the field of the “attribute_of_groupID” is unnecessary. - In addition, in the above-described embodiment, an example has been shown in which the plurality of group encoded data includes both of the channel encoded data and the object encoded data (see
FIG. 3 ). However, the present technology can be applied similarly also to a case in which the plurality of group encoded data includes only the channel encoded data, or includes only the object encoded data. - In addition, in the above-described embodiment, an example has been shown in which the container is the transport stream (MPEG-2 TS). However, the present technology can be applied similarly also to a system in which delivery is performed by the container of MP4 or another format. For example, it is an MPEG-DASH based stream delivery system, or a transmission/reception system dealing with an MPEG Media Transport (MMT) structure transmission stream.
- Incidentally, the present technique may also be embodied in the structures described below.
- (1) A transmission device including:
- a transmission unit for transmitting a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data; and
- an information insertion unit for inserting attribute information indicating an attribute of each of the plurality of group encoded data, into a layer of the container.
- (2) The transmission device according to the (1), wherein
- the information insertion unit further inserts stream correspondence information indicating an audio stream including each of the plurality of group encoded data, into the layer of the container.
- (3) The transmission device according the (2), wherein
- the stream correspondence information is information indicating a correspondence between a group identifier for identifying each of the plurality of group encoded data and a stream identifier for identifying each of the predetermined number of audio streams.
- (4) The transmission device according to the (3), wherein
- the information insertion unit further inserts stream identifier information indicating a stream identifier of each of the predetermined number of audio streams, into the layer of the container.
- (5) The transmission device according to the (4), wherein
- the container is an MPEG2-TS, and
- the information insertion unit inserts the stream identifier information into an audio elementary stream loop corresponding to each of the predetermined number of audio streams existing under a program map table.
- (6) The transmission device according the (2), wherein
- the stream correspondence information is information indicating a correspondence between the group identifier for identifying each of the plurality of group encoded data and a packet identifier to be attached during packetizing of each of the predetermined number of audio streams.
- (7) The transmission device according the (2), wherein
- the stream correspondence information is information indicating a correspondence between the group identifier for identifying each of the plurality of group encoded data and type information indicating a stream type of each of the predetermined number of audio streams.
- (8) The transmission device according to any of the (2) to (7), wherein
- the container is an MPEG2-TS, and
- the information insertion unit inserts the attribute information and the stream correspondence information, into an audio elementary stream loop corresponding to any one audio stream of the predetermined number of audio streams existing under the program map table.
- (9) The transmission device according to any of the (1) to (8), wherein
- the plurality of group encoded data includes either or both of channel encoded data and object encoded data.
- (10) A transmission method including:
- a transmission step for transmitting a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, from a transmission unit; and
- an information insertion step for inserting attribute information indicating an attribute of each of the plurality of group encoded data, into a layer of the container.
- (11) A reception device including:
- a reception unit for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container; and
- a processing unit for processing the predetermined number of audio streams included in the container received, on the basis of the attribute information.
- (12) The reception device according to the (11), wherein
- stream correspondence information indicating an audio stream including each of the plurality of group encoded data is further inserted into the layer of the container, and
- the processing unit processes the predetermined number of audio streams on the basis of the stream correspondence information, besides the attribute information.
- (13) The reception device according to the (12), wherein
- the processing unit selectively performs decoding processing to an audio stream including group encoded data holding an attribute conforming to a speaker configuration and user selection information, on the basis of the attribute information and the stream correspondence information.
- (14) The reception device according to any of the (11) to (13), wherein
- the plurality of group encoded data includes either or both of channel encoded data and object encoded data.
- (15) A reception method including:
- a reception step for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, by a reception unit, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container; and
- a processing step for processing the predetermined number of audio streams included in the container received, on the basis of the attribute information.
- (16) A reception device including:
- a reception unit for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container;
- a processing unit for selectively acquiring predetermined group encoded data on the basis of the attribute information from the predetermined number of audio streams included in the container received, and reconfiguring an audio stream including the predetermined group encoded data; and
- a stream transmission unit for transmitting the audio stream reconfigured in the processing unit to an external device.
- (17) The reception device according to the (16), wherein
- stream correspondence information indicating an audio stream including each of the plurality of group encoded data is further inserted into the layer of the container, and
- the processing unit selectively acquires the predetermined group encoded data from the predetermined number of audio streams on the basis of the stream correspondence information, besides the attribute information.
- (18) A reception method including:
- a reception step for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, by a reception unit, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container;
- a processing step for selectively acquiring predetermined group encoded data on the basis of the attribute information from the predetermined number of audio streams included in the container received, and reconfiguring an audio stream including the predetermined group encoded data; and
- a stream transmission step for transmitting the audio stream reconfigured in the processing step to an external device.
- The main feature of the present technology is that the processing load of the reception side can be reduced by inserting the attribute information indicating the attribute of each of the plurality of group encoded data included in the predetermined number of audio streams and the stream correspondence information indicating the audio stream including each of the plurality of group encoded data, into the layer of the container (see
FIG. 13 ). -
- 10 Transmission/reception system
- 100 Service transmitter
- 110 Stream generation unit
- 112 Video encoder
- 113 Audio encoder
- 114 Multiplexer
- 200, 200A Service receiver
- 201 Reception unit
- 202 Demultiplexer
- 203 Video decoder
- 204 Video processing circuit
- 205 Panel drive circuit
- 206 Display panel
- 211-1 to 211-N Multiplexing buffer
- 212 Combiner
- 213 3D audio decoder
- 214 Audio output processing circuit
- 215 Speaker system
- 221 CPU
- 222 Flash ROM
- 223 DRAM
- 224 Internal bus
- 225 Remote control reception unit
- 226 Remote control transmitter
- 231 Stream reconfiguration unit
- 232 Delivery interface
- 300 Device
Claims (18)
1. A transmission device comprising:
a transmission unit for transmitting a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data; and
an information insertion unit for inserting attribute information indicating an attribute of each of the plurality of group encoded data, into a layer of the container.
2. The transmission device according to claim 1 , wherein
the information insertion unit further inserts stream correspondence information indicating an audio stream including each of the plurality of group encoded data, into the layer of the container.
3. The transmission device according to claim 2 , wherein
the stream correspondence information is information indicating a correspondence between a group identifier for identifying each of the plurality of group encoded data and a stream identifier for identifying each of the predetermined number of audio streams.
4. The transmission device according to claim 3 , wherein
the information insertion unit further inserts stream identifier information indicating a stream identifier of each of the predetermined number of audio streams, into the layer of the container.
5. The transmission device according to claim 4 , wherein
the container is an MPEG2-TS, and
the information insertion unit inserts the stream identifier information into an audio elementary stream loop corresponding to each of the predetermined number of audio streams existing under a program map table.
6. The transmission device according to claim 2 , wherein
the stream correspondence information is information indicating a correspondence between the group identifier for identifying each of the plurality of group encoded data and a packet identifier to be attached during packetizing of each of the predetermined number of audio streams.
7. The transmission device according to claim 2 , wherein
the stream correspondence information is information indicating a correspondence between the group identifier for identifying each of the plurality of group encoded data and type information indicating a stream type of each of the predetermined number of audio streams.
8. The transmission device according to claim 2 , wherein
the container is an MPEG2-TS, and
the information insertion unit inserts the attribute information and the stream correspondence information, into an audio elementary stream loop corresponding to any one audio stream of the predetermined number of audio streams existing under the program map table.
9. The transmission device according to claim 1 , wherein
the plurality of group encoded data includes either or both of channel encoded data and object encoded data.
10. A transmission method comprising:
a transmission step for transmitting a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, from a transmission unit; and
an information insertion step for inserting attribute information indicating an attribute of each of the plurality of group encoded data, into a layer of the container.
11. A reception device comprising:
a reception unit for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container; and
a processing unit for processing the predetermined number of audio streams included in the container received, on the basis of the attribute information.
12. The reception device according to claim 11 , wherein
stream correspondence information indicating an audio stream including each of the plurality of group encoded data is further inserted into the layer of the container, and
the processing unit processes the predetermined number of audio streams on the basis of the stream correspondence information, besides the attribute information.
13. The reception device according to claim 12 , wherein
the processing unit selectively performs decoding processing to an audio stream including group encoded data holding an attribute conforming to a speaker configuration and user selection information, on the basis of the attribute information and the stream correspondence information.
14. The reception device according to claim 11 , wherein
the plurality of group encoded data includes either or both of channel encoded data and object encoded data.
15. A reception method comprising:
a reception step for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, by a reception unit, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container; and
a processing step for processing the predetermined number of audio streams included in the container received, on the basis of the attribute information.
16. A reception device comprising:
a reception unit for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container;
a processing unit for selectively acquiring predetermined group encoded data on the basis of the attribute information from the predetermined number of audio streams included in the container received, and reconfiguring an audio stream including the predetermined group encoded data; and
a stream transmission unit for transmitting the audio stream reconfigured in the processing unit to an external device.
17. The reception device according to claim 16 , wherein
stream correspondence information indicating an audio stream including each of the plurality of group encoded data is further inserted into the layer of the container, and
the processing unit selectively acquires the predetermined group encoded data from the predetermined number of audio streams on the basis of the stream correspondence information, besides the attribute information.
18. A reception method comprising:
a reception step for receiving a predetermined format container having a predetermined number of audio streams including a plurality of group encoded data, by a reception unit, attribute information indicating an attribute of each of the plurality of group encoded data being inserted into a layer of the container;
a processing step for selectively acquiring predetermined group encoded data on the basis of the attribute information from the predetermined number of audio streams included in the container received, and reconfiguring an audio stream including the predetermined group encoded data; and
a stream transmission step for transmitting the audio stream reconfigured in the processing step to an external device.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/307,605 US20230260523A1 (en) | 2014-09-04 | 2023-04-26 | Transmission device, transmission method, reception device and reception method |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014180592 | 2014-09-04 | ||
JP2014-180592 | 2014-09-04 | ||
PCT/JP2015/074593 WO2016035731A1 (en) | 2014-09-04 | 2015-08-31 | Transmitting device, transmitting method, receiving device and receiving method |
US201715505782A | 2017-02-22 | 2017-02-22 | |
US18/307,605 US20230260523A1 (en) | 2014-09-04 | 2023-04-26 | Transmission device, transmission method, reception device and reception method |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/074593 Continuation WO2016035731A1 (en) | 2014-09-04 | 2015-08-31 | Transmitting device, transmitting method, receiving device and receiving method |
US15/505,782 Continuation US11670306B2 (en) | 2014-09-04 | 2015-08-31 | Transmission device, transmission method, reception device and reception method |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230260523A1 true US20230260523A1 (en) | 2023-08-17 |
Family
ID=55439793
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/505,782 Active US11670306B2 (en) | 2014-09-04 | 2015-08-31 | Transmission device, transmission method, reception device and reception method |
US18/307,605 Pending US20230260523A1 (en) | 2014-09-04 | 2023-04-26 | Transmission device, transmission method, reception device and reception method |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/505,782 Active US11670306B2 (en) | 2014-09-04 | 2015-08-31 | Transmission device, transmission method, reception device and reception method |
Country Status (6)
Country | Link |
---|---|
US (2) | US11670306B2 (en) |
EP (3) | EP3799044B1 (en) |
JP (4) | JP6724782B2 (en) |
CN (2) | CN106796793B (en) |
RU (1) | RU2698779C2 (en) |
WO (1) | WO2016035731A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2701060C2 (en) * | 2014-09-30 | 2019-09-24 | Сони Корпорейшн | Transmitting device, transmission method, receiving device and reception method |
CN107210041B (en) * | 2015-02-10 | 2020-11-17 | 索尼公司 | Transmission device, transmission method, reception device, and reception method |
US10027994B2 (en) * | 2016-03-23 | 2018-07-17 | Dts, Inc. | Interactive audio metadata handling |
CN110945848B (en) * | 2017-08-03 | 2022-04-15 | 安步拓科技股份有限公司 | Client device, data collection system, data transmission method, and program |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5956674A (en) * | 1995-12-01 | 1999-09-21 | Digital Theater Systems, Inc. | Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels |
JP4393435B2 (en) * | 1998-11-04 | 2010-01-06 | 株式会社日立製作所 | Receiver |
JP2000181448A (en) | 1998-12-15 | 2000-06-30 | Sony Corp | Device and method for transmission, device and method for reception, and provision medium |
US6885987B2 (en) * | 2001-02-09 | 2005-04-26 | Fastmobile, Inc. | Method and apparatus for encoding and decoding pause information |
JP3382235B2 (en) * | 2001-10-05 | 2003-03-04 | 株式会社東芝 | Still image information management system |
EP1427252A1 (en) * | 2002-12-02 | 2004-06-09 | Deutsche Thomson-Brandt Gmbh | Method and apparatus for processing audio signals from a bitstream |
US7742683B2 (en) * | 2003-01-20 | 2010-06-22 | Pioneer Corporation | Information recording medium, information recording device and method, information reproduction device and method, information recording/reproduction device and method, computer program for controlling recording or reproduction, and data structure containing control signal |
WO2005076622A1 (en) | 2004-02-06 | 2005-08-18 | Sony Corporation | Information processing device, information processing method, program, and data structure |
KR20070007824A (en) * | 2004-03-17 | 2007-01-16 | 엘지전자 주식회사 | Recording medium, method, and apparatus for reproducing text subtitle streams |
US8131134B2 (en) * | 2004-04-14 | 2012-03-06 | Microsoft Corporation | Digital media universal elementary stream |
DE102004046746B4 (en) * | 2004-09-27 | 2007-03-01 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method for synchronizing additional data and basic data |
KR100754197B1 (en) * | 2005-12-10 | 2007-09-03 | 삼성전자주식회사 | Video service providing and receiving method in DAB system, and apparatus thereof |
US9178535B2 (en) * | 2006-06-09 | 2015-11-03 | Digital Fountain, Inc. | Dynamic stream interleaving and sub-stream based delivery |
JP4622950B2 (en) * | 2006-07-26 | 2011-02-02 | ソニー株式会社 | RECORDING DEVICE, RECORDING METHOD, RECORDING PROGRAM, IMAGING DEVICE, IMAGING METHOD, AND IMAGING PROGRAM |
EP2047669B1 (en) * | 2006-07-28 | 2014-05-21 | Unify GmbH & Co. KG | Method for carrying out an audio conference, audio conference device, and method for switching between encoders |
CN1971710B (en) * | 2006-12-08 | 2010-09-29 | 中兴通讯股份有限公司 | Single-chip based multi-channel multi-voice codec scheduling method |
JP2008199528A (en) * | 2007-02-15 | 2008-08-28 | Sony Corp | Information processor, information processing method, program, and program storage medium |
EP2083585B1 (en) * | 2008-01-23 | 2010-09-15 | LG Electronics Inc. | A method and an apparatus for processing an audio signal |
KR101461685B1 (en) * | 2008-03-31 | 2014-11-19 | 한국전자통신연구원 | Method and apparatus for generating side information bitstream of multi object audio signal |
CN101572087B (en) * | 2008-04-30 | 2012-02-29 | 北京工业大学 | Method and device for encoding and decoding embedded voice or voice-frequency signal |
US8745502B2 (en) * | 2008-05-28 | 2014-06-03 | Snibbe Interactive, Inc. | System and method for interfacing interactive systems with social networks and media playback devices |
CN102100009B (en) * | 2008-07-15 | 2015-04-01 | Lg电子株式会社 | A method and an apparatus for processing an audio signal |
WO2010008198A2 (en) * | 2008-07-15 | 2010-01-21 | Lg Electronics Inc. | A method and an apparatus for processing an audio signal |
US8588947B2 (en) * | 2008-10-13 | 2013-11-19 | Lg Electronics Inc. | Apparatus for processing an audio signal and method thereof |
US8768388B2 (en) | 2009-04-09 | 2014-07-01 | Alcatel Lucent | Method and apparatus for UE reachability subscription/notification to facilitate improved message delivery |
RU2409897C1 (en) * | 2009-05-18 | 2011-01-20 | Самсунг Электроникс Ко., Лтд | Coder, transmitting device, transmission system and method of coding information objects |
CN102667921B (en) * | 2009-10-20 | 2014-09-10 | 弗兰霍菲尔运输应用研究公司 | Audio encoder, audio decoder, method for encoding an audio information, method for decoding an audio information |
CN102668581A (en) * | 2009-10-25 | 2012-09-12 | Lg电子株式会社 | Method for processing broadcast program information and broadcast receiver |
US9456234B2 (en) * | 2010-02-23 | 2016-09-27 | Lg Electronics Inc. | Broadcasting signal transmission device, broadcasting signal reception device, and method for transmitting/receiving broadcasting signal using same |
US9143271B2 (en) * | 2010-04-01 | 2015-09-22 | Lg Electronics Inc. | Broadcasting signal transmitting apparatus, broadcast signal receiving apparatus, and broadcast signal transceiving method in a broadcast signal transceiving apparatus |
JP5594002B2 (en) | 2010-04-06 | 2014-09-24 | ソニー株式会社 | Image data transmitting apparatus, image data transmitting method, and image data receiving apparatus |
CN102222505B (en) * | 2010-04-13 | 2012-12-19 | 中兴通讯股份有限公司 | Hierarchical audio coding and decoding methods and systems and transient signal hierarchical coding and decoding methods |
JP5577823B2 (en) * | 2010-04-27 | 2014-08-27 | ソニー株式会社 | Transmitting apparatus, transmitting method, receiving apparatus, and receiving method |
JP5652642B2 (en) * | 2010-08-02 | 2015-01-14 | ソニー株式会社 | Data generation apparatus, data generation method, data processing apparatus, and data processing method |
JP2012244411A (en) * | 2011-05-19 | 2012-12-10 | Sony Corp | Image data transmission apparatus, image data transmission method and image data reception apparatus |
AU2012279349B2 (en) | 2011-07-01 | 2016-02-18 | Dolby Laboratories Licensing Corporation | System and tools for enhanced 3D audio authoring and rendering |
JP2013090016A (en) * | 2011-10-13 | 2013-05-13 | Sony Corp | Transmitter, transmitting method, receiver and receiving method |
US9769488B2 (en) * | 2012-02-02 | 2017-09-19 | Sun Patent Trust | Methods and apparatuses for 3D media data generation, encoding, decoding and display using disparity information |
WO2013161442A1 (en) * | 2012-04-24 | 2013-10-31 | ソニー株式会社 | Image data transmission device, image data transmission method, image data reception device, and image data reception method |
CN103748629B (en) * | 2012-07-02 | 2017-04-05 | 索尼公司 | Decoding apparatus and method, code device and method and program |
US9860458B2 (en) * | 2013-06-19 | 2018-01-02 | Electronics And Telecommunications Research Institute | Method, apparatus, and system for switching transport stream |
KR102273757B1 (en) * | 2014-01-03 | 2021-07-06 | 엘지전자 주식회사 | Apparatus for transmitting broadcast signals, apparatus for receiving broadcast signals, method for transmitting broadcast signals and method for receiving broadcast signals |
CN106068651B (en) * | 2014-03-18 | 2020-10-16 | 皇家飞利浦有限公司 | Audio-visual content item data stream |
EP3522554B1 (en) * | 2014-05-28 | 2020-12-02 | FRAUNHOFER-GESELLSCHAFT zur Förderung der angewandten Forschung e.V. | Data processor and transport of user control data to audio decoders and renderers |
-
2015
- 2015-08-31 EP EP20208155.0A patent/EP3799044B1/en active Active
- 2015-08-31 CN CN201580045713.2A patent/CN106796793B/en active Active
- 2015-08-31 EP EP15838724.1A patent/EP3196876B1/en active Active
- 2015-08-31 WO PCT/JP2015/074593 patent/WO2016035731A1/en active Application Filing
- 2015-08-31 RU RU2017106022A patent/RU2698779C2/en active
- 2015-08-31 EP EP23216185.1A patent/EP4318466A3/en active Pending
- 2015-08-31 US US15/505,782 patent/US11670306B2/en active Active
- 2015-08-31 JP JP2016546628A patent/JP6724782B2/en active Active
- 2015-08-31 CN CN202010846670.0A patent/CN111951814A/en active Pending
-
2020
- 2020-06-25 JP JP2020109929A patent/JP6908168B2/en active Active
-
2021
- 2021-07-01 JP JP2021110252A patent/JP7238925B2/en active Active
-
2023
- 2023-03-01 JP JP2023030769A patent/JP2023085253A/en active Pending
- 2023-04-26 US US18/307,605 patent/US20230260523A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
WO2016035731A1 (en) | 2016-03-10 |
JP2020182221A (en) | 2020-11-05 |
CN106796793B (en) | 2020-09-22 |
CN111951814A (en) | 2020-11-17 |
EP3196876A1 (en) | 2017-07-26 |
JP2021177638A (en) | 2021-11-11 |
JP6724782B2 (en) | 2020-07-15 |
US20170249944A1 (en) | 2017-08-31 |
RU2017106022A3 (en) | 2019-03-26 |
JP2023085253A (en) | 2023-06-20 |
EP4318466A3 (en) | 2024-03-13 |
JP6908168B2 (en) | 2021-07-21 |
JPWO2016035731A1 (en) | 2017-06-15 |
US11670306B2 (en) | 2023-06-06 |
CN106796793A (en) | 2017-05-31 |
EP3799044B1 (en) | 2023-12-20 |
RU2017106022A (en) | 2018-08-22 |
EP3196876B1 (en) | 2020-11-18 |
RU2698779C2 (en) | 2019-08-29 |
JP7238925B2 (en) | 2023-03-14 |
EP3196876A4 (en) | 2018-03-21 |
EP3799044A1 (en) | 2021-03-31 |
EP4318466A2 (en) | 2024-02-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20230260523A1 (en) | Transmission device, transmission method, reception device and reception method | |
US10142757B2 (en) | Transmission device, transmission method, reception device, and reception method | |
US20240114202A1 (en) | Transmission apparatus, transmission method, reception apparatus and reception method for transmitting a plurality of types of audio data items | |
US10878828B2 (en) | Transmission device, transmission method, reception device, and reception method | |
US10614823B2 (en) | Transmitting apparatus, transmitting method, receiving apparatus, and receiving method | |
JPWO2017104519A1 (en) | Transmitting apparatus, transmitting method, receiving apparatus, and receiving method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |