Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/845—Structuring of content, e.g. decomposing content into time segments
  • H04N21/8451—Structuring of content, e.g. decomposing content into time segments using Advanced Video Coding [AVC]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
  • H04N21/2368—Multiplexing of audio and video streams
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04H—BROADCAST COMMUNICATION
  • H04H20/00—Arrangements for broadcast or for distribution combined with broadcast
  • H04H20/44—Arrangements characterised by circuits or components specially adapted for broadcast
  • H04H20/46—Arrangements characterised by circuits or components specially adapted for broadcast specially adapted for broadcast systems covered by groups H04H20/53-H04H20/95
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04H—BROADCAST COMMUNICATION
  • H04H20/00—Arrangements for broadcast or for distribution combined with broadcast
  • H04H20/65—Arrangements characterised by transmission systems for broadcast
  • H04H20/71—Wireless systems
  • H04H20/72—Wireless systems of terrestrial networks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04H—BROADCAST COMMUNICATION
  • H04H20/00—Arrangements for broadcast or for distribution combined with broadcast
  • H04H20/86—Arrangements characterised by the broadcast information itself
  • H04H20/88—Stereophonic broadcast systems
  • H04H20/89—Stereophonic broadcast systems using three or more audio channels, e.g. triphonic or quadraphonic
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04H—BROADCAST COMMUNICATION
  • H04H40/00—Arrangements specially adapted for receiving broadcast information
  • H04H40/18—Arrangements characterised by circuits or components specially adapted for receiving
  • H04H40/27—Arrangements characterised by circuits or components specially adapted for receiving specially adapted for broadcast systems covered by groups H04H20/53 - H04H20/95
  • H04H40/36—Arrangements characterised by circuits or components specially adapted for receiving specially adapted for broadcast systems covered by groups H04H20/53 - H04H20/95 specially adapted for stereophonic broadcast receiving
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/167—Synchronising or controlling image signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/194—Transmission of image signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
  • H04N19/25—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with scene description coding, e.g. binary format for scenes [BIFS] compression
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/23412—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/234318—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into objects, e.g. MPEG-4 objects
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
  • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04H—BROADCAST COMMUNICATION
  • H04H20/00—Arrangements for broadcast or for distribution combined with broadcast
  • H04H20/28—Arrangements for simultaneous broadcast of plural pieces of information

Definitions

• the present invention relates to a 3-dimensional Digital Multimedia Broadcasting (3D DMB) system; and more particularly, to 3D DMB transmitting and receiving systems for transmitting a 3D Audio/Video (AV) by encoding the 3D AV and providing a service to a user by decoding the received signal into the 3D AV to provide a more realistic stereophonic broadcasting to a user while maintaining compatibility with a conventional DMB system.
• 3D DMB 3-dimensional Digital Multimedia Broadcasting
• AV 3D Audio/Video
• a field of communication among people has been remarkably changed due to development of a computer and a communication technology, and a two-way service, is actively used instead of a conventional one-way service by connecting diverse data of diverse multimedia to mobile technology such as a Digital Multimedia Broadcasting (DMB) .
• DMB Digital Multimedia Broadcasting
• the DMB system is a broadcasting system for providing diverse multimedia services such as a video, an audio and data to a mobile user anytime and anywhere, and a service of the DMB system has been prepared in Korea for the first time in the world.
• the DMB system which is a transmission system, utilizes a Digital Audio Broadcasting (DAB) as a transmission system, uses MPEG-4 as a media process method, and uses MPEG-4 and MPEG-2 system as a specification for multiplexing, synchronizing and transmitting media data.
• DMB Digital Audio Broadcasting
• MPEG-4 as a media process method
• MPEG-4 and MPEG-2 system as a specification for multiplexing, synchronizing and transmitting media data.
• a user can be provided a high-quality audio and video service of a CD level by using the DMB system.
• the user demand for a stereophonic sound in digital broadcasting is increasing due to rapid propagation of a multi-channel sound audio and development of a 3D audio acquisition and restoring technology.
• the 3D AV is used in diverse application fields such as sports relay, advertisement, education, medical service and game due to increasing interest and request with respect to the 3D AV, but a DMB service has a ⁇ problem that the sense of real and 3D effect is very low since the DMB service is provided with a focus on 2D AV.
• an object of the present invention to provide a three dimensional (3D) digital multimedia broadcasting system (DMB) for providing a 3D Audio/Video (AV) service, which maintains comparability with a conventional DMB system and provides a more realistic 3D AV service to a user, by processing a binocular 3D moving picture and 3D sound by using a conventional system structure.
• DMB digital multimedia broadcasting system
• AV Audio/Video
• a 3-dimensional digital multimedia broadcasting (3D DMB) transmitting system including: a video encoder for receiving and encoding a video signal of a binocular 3D image; an audio encoder for receiving and encoding an audio signal of a 3D sound; a packetizer for packetizing the encoded video and audio signals as a sync layer (SL) packet; and a multiplexer for transforming and multiplexing the SL packet, which is packetized by the packetizer and outputting the multiplexed SL packet.
• 3D DMB 3-dimensional digital multimedia broadcasting
• a three dimensional digital multimedia broadcasting (3D DMB) receiving system including: a demultiplexer for outputting an SL packet by demultiplexing a stream including binocular 3D image information and 3D sound information; a depacketizer for depacketizing the SL packet and outputting an encoding streams of a binocular 3D video signal and a 3D sound audio signal; a video decoder for decoding an encoding stream with respect to the binocular 3D video signal; an audio decoder for decoding a encoding stream with respect to the 3D sound audio signal; and a scene restorer for forming and outputting a 2D audio/video (AV) or 3D AV scene by using a video/3D additional video signal and an audio/3D additional audio signal which are inputted from the video decoder and the audio decoder.
• 3D DMB three dimensional digital multimedia broadcasting
• Fig. 1 is a block diagram showing a three-dimensional digital multimedia broadcasting (3D DMB) transmitting system in accordance with an embodiment of the present invention
• Fig. 2 is a block diagram illustrating a video encoding module of Fig. 1;
• Fig. 3 is a block diagram illustrating a 3D additional video encoding module of Fig. 1;
• Fig. 4 is a block diagram illustrating an audio encoding module of Fig. 1;
• Fig. 5 is a 3D additional audio encoding module of Fig.
• Fig. 6 is a block diagram illustrating an additional audio pre-processor of Fig. 5;
• Fig. 7 is a block diagram illustrating a system encoding module of Fig. 1;
• Fig. 8 is a block diagram showing a data structure of an object descriptor (OD) in accordance with an embodiment of the present invention
• Fig. 9 is a block diagram illustrating an M4 over M2 module of Fig. 1;
• Fig. 10 is a block diagram showing a 3D digital multimedia broadcasting receiving system in accordance with an embodiment of the present invention
• Fig. 11 is a block diagram illustrating an M2 over M4 module of Fig. 10;
• Fig. 12 is a block diagram illustrating a system analyzing module of Fig. 10;
• Fig. 13 is a block diagram illustrating a 3D video decoding module of Fig. 10;
• Fig. 14 is a block diagram of a 3D audio decoding module of Fig. 10;
• Fig. 15 is a block diagram illustrating a 3D audio post-processor of Fig. 14.
• Fig. 16 is a block diagram illustrating a scene generating module of Fig. 10.
• Fig. 1 is a block diagram showing a three-dimensional digital multimedia broadcasting (3D DMB) transmitting system in accordance with an embodiment of the present invention.
• the 3D digital multimedia broadcasting transmitting system of the present invention is a system for receiving an Audio/video (AV) data, i.e., a video signal and an audio signal, and 3D additional AV data, i.e., a 3D additional video signal and a 3D additional audio signal, encoding the AV data and the 3D additional AV data based on MPEG-4, and outputting multiplexed MPEG-2 Transport Stream (TS) .
• AV Audio/video
• 3D additional AV data i.e., a 3D additional video signal and a 3D additional audio signal
• TS MPEG-2 Transport Stream
• the 3D digital multimedia broadcasting transmitting system includes a video encoding module 110, a 3D additional video encoding module 130, an audio encoding module 120, a 3D additional audio encoding module 140, a system encoding module 150 and an M4 over M2 module 160.
• the video encoding module 110 is a module for encoding one of a left image and a right image of a binocular 3D moving picture as an MPEG-4 part 10 Advanced Video Coding (AVC) baseline profile specification.
• the audio encoding module 120 is a module for encoding an audio signal transmitted from an outside as an MPEG-4 Bit Sliced Arithmetic Coding (BSAC) specification.
• BSAC MPEG-4 Bit Sliced Arithmetic Coding
• the 3D additional audio encoding module 140 is a module for encoding the 3D additional audio signal inputted from the outside as the MPEG-4 Bit BSAC specification.
• the system encoding module 150 generates, encodes MPEG-4 Initial Object Descriptor (IOD) /OD/Binary Format for Scene (BIFS) data and packetizes an object descriptor (OD) / Binary Format for scenes (BIFS) stream generated by using Elementary Stream (ES) outputted from the 4 decoding modules 110 to 140 and the data received from the outside into a sync layer (SL) packet.
• IOD Initial Object Descriptor
• BIFS Binary Format for scenes
• ES Elementary Stream
• the M4 over M2 module 160 receives SL packet data, packetizes the SL packet data into MPEG-2 TSs, multiplexes the packetized MPEG-2 TSs into one MPEG-2 TS and outputs the MPEG-2 TS.
• Fig. 2 is a block diagram illustrating a video encoding module of Fig. 1.
• the video encoding module 110 includes a video pre-processor 111 and a video encoder 113.
• the video pre-processor 111 receives a left or right video signal with respect to a binocular 3D moving picture from an outside and transforms a size of the image into a size of Quater Video Graphic Array (QVGA) 320X240 or Common Intermediate Format (CIF) 352X288.
• the video encoder 113 outputs a video Elementary Stream (ES) , which is obtained by encoding a video signal, a size of which is transformed in the additional video pre-processor 111 and the MPEG-4 part 10 Advanced Video Coding (AVC) baseline profile specification and an encoding parameter, which is a bit rate.
• the video encoding module 110 can receive and process a general 2D moving picture video signal.
• Fig. 3 is a block diagram illustrating a 3D additional video encoding module of Fig. 1.
• the 3D additional video encoding module 130 includes an additional video pre-processor 131 and an additional video encoder 133.
• the additional video pre-processor 131 receives a right or left video signal with respect to a binocular 3D moving picture from an outside and transforms a size of the image into a size of 160X240/320X120 or 176X288/352X144, which is reduced as large as a half of the QVGA/CIF size in height or width.
• the additional video encoder 133 outputs a 3D additional video ES, which is encoding a video signal transformed in the additional video pre-processor 131 into the MPEG-4 part 10 AVC baseline profile specification and an encoding parameter, which is a bit rate.
• Fig. 4 is a block diagram illustrating an audio encoding module of Fig. 1.
• the audio encoding module 120 includes an audio pre-processor 121 and an audio encoder 123.
• the audio pre-processor 121 transforms resolution and a sampling frequency of an audio signal received from the outside to meet an MPEG-4 BSAC specification.
• the audio encoder 123 receives an audio signal transformed in the audio pre-processor 121 and a 2 channel downmix audio signal generated in the 3D additional audio encoding module 140, and also generates an audio ES, which is encoded into an MPEG-4 BSAC specification based on an encoding parameter, which is a bit rate.
• Fig. 5 is a 3D additional audio encoding module of Fig. 1 in accordance with an embodiment of the present invention.
• the 3D additional audio encoding module 140 includes an additional audio pre-processor 141 and an additional audio encoder 143.
• the additional audio pre-processor 141 generates and outputs a 2 channel downmix signal and 3 channel 3D additional audio signal by using a 3D additional audio signal received from the outside, wherein the 2 channel downmix signal is outputted to the audio encoder 123 of the MPEG-4 audio encoding module 120.
• the additional audio encoder 143 outputs a 3D additional audio ES by encoding the 3 channel 3D additional audio signal into an MPEG-4 BSAC specification.
• the 3D additional audio signal can be diverse multi-channel stereophonic signal such as a 5 channel stereophonic signal, which includes a forward left speaker signal L, a forward right speaker signal R, a forward central speaker signal C, a backward left speaker signal LS and a backward right speaker signal RS, or 5.1 channel stereophonic signal, in which Sub-Woofer signal SW is added to the 5 signals.
• Fig. 6 is a block diagram illustrating an additional audio pre-processor of Fig. 5.
• the additional audio pre ⁇ processing block 141 includes a pre-processor 145 and a channel-mixer 147.
• the pre-processor 145 receives a 3D additional audio signal, which is diverse channel stereophonic signals, from the outside, processes a signal characteristic such as a sampling frequency and a number of quantization bit, and generates 5 channel stereophonic signal in which low-pass filtering is performed.
• the channel-mixer 147 generates a 2 channel downmix signals LO and RO, and 3 channel additional audio signals T, Ql and Q2 by using a mixing method defined by an ITU-R BS.775-1 specification from the 5 channel stereophonic signal.
• the 2 channel downmix signals LO and RO are left/right audio signals used when a stereo is played
• the 3 channel additional audio signal T, Ql and Q2 are signals for reorganizing an original signal.
• the pre-processor 145 can output the inputted signal without any special process.
• Fig. 7 is a block diagram illustrating a system encoding module of Fig. 1.
• the system encoding module 150 includes an SL packetizer 151, an OD/BIFS generator 153 and Initial Object Descriptor (IOD) generator 155.
• IOD Initial Object Descriptor
• the OD/BIFS generator 153 generates and outputs an OD/BIFS ES by using an OD/BIFS text data received from the outside.
• a 3D AV is expressed by individually using one OD with respect to a video/3D additional video and an audio/3D additional audio.
• the SL packetizer 151 receives a video ES, an audio ES, a 3D additional video ES, a 3D additional audio ES and the OD/BIFS ES from the above-mentioned 4 encoding modules, and outputs them by packetizing each of them as an SL packet according to a Korean mobile multimedia broadcasting specification.
• the IOD generator 155 generates and outputs an IOD ES by using the IOD text data transmitted from the outside.
• Fig. 8 is a block diagram showing a data structure of an object descriptor (OD) in accordance with an embodiment of the present invention.
• two ES_Descriptors for left or right video and a 3D additional video of a right or left video are subordinately included in one OD to express a 3D AV while maintaining comparability with an OD/BIFS structure of a conventional DMB system.
• two ES_Descriptors for one OD lower audio and a 3D additional audio have a subordinate relationship.
• the subordinate relationship can be expressed by using Stream DependenceFlag of MPEG-4 OD, and depends0n_ES_ID.
• ObjectTypelndication of a 3D additional video ES is set as User private, and stream type is set as Visual to provide a service of a 3D AV while maintaining comparability with a conventional DMB system.
• ObjectTypelndication of a 3D additional audio ES is set as User private and stream type is set as Audio.
• Other video and audio ES are set according to a conventional DMB system specification.
• Fig. 9 is a block diagram illustrating an M4 over M2 module of Fig. 1.
• the M4 over M2 module 160 includes a Packetized Elementary Stream (PES) Packetizer 161, a PES to TS Packetizer 162, a multiplexer 163, a switch 164, a 14496 section packetizer 165, a PSI generator 166, a Program Specific Information PSI section to TS Packetizer 167 and a 14496 section to TS Packetizer 168.
• PES Packetized Elementary Stream
• the switch 164 receives an OD/BIFS SL packet from the system encoding module 150 and outputs the OD/BIFS SL packet as the PES Packetizer 161 or the 14496 section packetizer 165 according to encoding information of an SL packet.
• the video SL packet, the Audio SL packet, the 3D additional video SL packet, the 3D additional audio SL packet received from the system encoding module 150, and the OD/BIFS SL packet inputted from the switch 164 are individuallly packetized as PES and outputted to the PES to TS Packetizer 162 by the PES packetizer 161.
• the OD/BIFS SL packet inputted from the switch 164 is packetized into a 14496 section and outputted to the 14496 section to TS Packetizer 168 by the 14496 section packetizer 165.
• the PSI generator 166 generates PSI including Program Association Table (PMT) section (PA_section) and Program Map Table (PMT) section (PM_section) by using the IOD information received from the system encoding module 150 and outputs the PSI to the PSI section to TS Packetizer 167.
• the inputted PES packet is packetized into MPEG-2 TS and outputted to the multiplexer 163 by the PES to TS Packetizer 162.
• the inputted 14496 seciton is packetized into MPEG-2 TS and outputted to the multiplexer 163 by the 14496 section to TS Packetizer 168.
• the PSI information is inputted and packetized into MPEG-2 TS and outputted to the multiplexer 163 by the PSI section to TS Packetizer 167.
• Fig. 10 is a block diagram showing a 3D digital multimedia broadcasting receiving system in accordance with an embodiment of the present invention.
• the 3D digital multimedia broadcasting receiving system of the present invention is a system for demultiplexing and decoding the multiplexed MPEG-2 TS, which is inputted from the outside, and displaying a 2D AV or a 3D AV to a user.
• the 3D digital multimedia broadcasting receiving system includes an M2 over M4 module 210, a system analyzing module 220, a 3D video decoding module 230, a 3D audio decoding module 240 and a scene generating module 250.
• the M2 over M4 module 210 outputs the video SL packet, the audio SL packet, the 3D additional video SL packet, the 3D additional audio SL packet, the OD/BIFS SL packet and the IOD data by demultiplexing and depacketizing the multiplexed MPEG-2 TS, which is inputted from an outside.
• the system analyzing module 220 depacketizes the received SL packet including the video SL packet, the Audio
• the SL packet, the 3D additional video SL packet, the 3D additional audio SL packet and the OD/BIFS SL packet outputs the SL packet as an encoding stream including a video ES, an audio ES, a 3D additional video ES and a 3D additional audio ES and outputs the IOD data and the
• the 3D video decoding module 230 outputs a video ES and a 3D additional video encoding ES as a 2D video scene or a 3D video scene by decoding the video ES and the 3D additional video encoding ES.
• the 3D audio decoding module 240 outputs a 3D audio signal as a 2D or 3D audio signal by decoding an audio ES and a 3D additional audio ES.
• the scene generating module 250 forms and outputs a scene according to a definition of the BIFS by using the
• Fig. 11 is a block diagram illustrating an M2 over M4 module of Fig. 10.
• the M2 over M4 module 210 includes a De_Multiplexer 211, a TS to PES de-packetizer
• the De_Multiplexer 211 individually demultiplexes the multiplexed MPEG-2 TS, which is inputted from the outside, and outputs the demultiplexed MPEG-2 TS as a single MPEG-2 TS with respect to a video ES, an audio ES, a 3D additional video ES, a 3D additional audio ES, OD/BIFS, a 14496 section and PSI.
• the TS to PES de-packetizer 212 receives an MPEG-2 TS of a video, an audio, a 3D additional video, a 3D additional audio and OD/BIFS from the De_Multiplexer 211, and outputs the MPEG-2 TS as a PES de-packetizer 215 by depacketizing the MPEG-2 TS into a PES packet.
• the TS to 14496 section de-packetizer 213 receives an MPEG-2 TS of the 14496 section and outputs the MPEG-2 TS to the 14496 section analyzer 216 by depacketizing the MPEG-2 TS into the 14496 section.
• the TS to PSI section de-packetizer 214 receives an
• the MPEG-2 TS of the PSI and outputs the MPEG-2 TS to the PSI section analyzer 217 by depacketizing the MPEG-2 TS into the PMP-section.
• the PES de-packetizer 215 receives each PES packet with respect to a video, an audio, a 3D additional video ES, a 3D additional audio and OD/BIFS from the TS to PES de- packetizer 212 and outputs a video SL packet, an audio SL packet, a 3D additional video SL packet, a 3D additional audio SL packet and an OD/BIFS SL packet by depacketizing the PES packets into an SL packet.
• the 14496 section analyzer 216 receives a 14496 section from the TS to 14496 section de-packetizer 213 and outputs the OD/BIFS SL packet by extracting the OD/BIFS SL packet.
• the PSI section analyzer 217 receives a PAT section and a PMT section and outputs IOD data by extracting the IOD data.
• Fig. 12 is a block diagram illustrating a system analyzing module of Fig. 10.
• the system analyzing module 220 includes a SL de-packetizer 221, an OD/BIFS decoder 222 and a IOD decoder 223.
• the SL de-packetizer 221 receives a video SL packet, an audio SL packet, a 3D additional video SL packet, a 3D additional audio SL packet and an OD/BIFS SL packet from the M2 over M4 module 210, depacketizes each of them into Encoding Stream and outputs the video ES and outputs the 3D additional video ES to the 3D video decoding module 230, the audio ES and 3D additional audio ES to the 3D audio decoding module 240, and the OD/BIFS data to the OD/BIFS decoder 222.
• the the OD/BIFS decoder 222 decodes the OD/BIFS encoding data transmitted from the SL de-packetizer 221 and outputs the decoded BIFS information to the scene generating module 250, wherein the decoded OD information is used to initialize each media decoder.
• the IOD decoder 223 outputs the IOD encoding data by receiving and decoding the IOD encoding data, and the decoded IOD data is used to extract OD/BIFS data.
• Fig. 13 is a block diagram illustrating a 3D video decoding module of Fig. 10.
• the 3D video decoding module 230 includes a video decoder 231, an additional video decoder 233 and a 3D video post-processor 235.
• the video decoder 231 receives a video ES of MPEG-4 part 10 AVC specification of 320X240/352X288 image size from the SL de-packetizer 221 of the system analyzing module 220 and outputs the video ES to the 3D video post- processor 235 by decoding the video ES.
• the additional video decoder 233 receives a 3D additional video ES of MPEG-4 part 10 AVC specification of 160X240/320X120 or 176X288/352X144 image size from the SL de-packetizer 221 of the system analyzing module 220 and outputs the 3D additional video ES to the 3D video post ⁇ processor 235 by decoding the 3D additional video ES.
• the 3D video post-processor 235 individually receives a decoded video signal and a 3D additional video signal form the video decoder 231 and the additional video decoder 233, and outputs the video signal and the 3D additional video signal as a 3D video signal of QUGA 320X240 or CIF 352X288 by synthesizing the signals according to a field when the video restoring mode information inputted from a user is a 3D video restoring mode, or outputs only a 2D video signal of 320X240 or 352X288 image size with ignoring the received 3D additional video signal when the video restoring mode information is a 2D video restoring mode.
• a user can enjoy 3D AV by selecting a desirable displaying method through this.
• Fig. 14 is a block diagram of a 3D audio decoding module of Fig. 10.
• the 3D audio decoding module 240 includes an audio decoder 241, an additional audio decoder 243 and a 3D audio post-processor 245.
• the audio decoder 241 receives an audio ES of an MPEG- 4 BSAC specification from the SL de-packetizer 221 of the system analyzing module 220 and outputs the audio ES to the 3D audio post-processor 245 by decoding the audio ES.
• the additional audio decoder 243 receives a 3D additional audio ES of an MPEG-4 BSAC specification from the SL de-packetizer 221 of the system analyzing module 220 and outputs the 3D additional audio ES to the 3D audio post-processor 245 by decoding the 3D additional audio ES.
• the 3D audio post-processor 245 receives a decoded audio signal and a 3D additional audio signal from the audio decoder 241 and the additional audio decoder 243, generates and outputs a 2D or 3D audio signal by using audio restoring information inputted from a user.
• Fig. 15 is a block diagram illustrating a 3D audio post-processor of Fig. 14.
• the 3D audio post-processor 245 includes a 2D/3D switch 246, a de-matrixer 247 and a virtual 3D combiner 248.
• the 2D/3D switch 246 outputs 5 channel 3D audio signals LO, RO, T, Ql and Q2 by synthesizing the decoded audio signal and the 3D additional audio signal when the audio restoring mode information received from a user is a
• the de-matrixer 247 receives an audio signal and a 3D audio signal, and outputs a 5 channel audio signals L, R, C, LS and RS by using ITU-R BS.775-1 specification.
• the Virtual 3D Synthesizer 248 receives and outputs the 5 channel audio signals by transforming the 5 channel audio signals into a 3D audio signal, which is formed of 2 channel virtual sound signals L and R.
• Fig. 16 is a block diagram illustrating a scene generating module of Fig. 10. As shown in Fig. 16, the scene generating module 250 includes a scene forming unit 251 and a renderer 253.
• the scene forming unit 251 receives AV data such as a 2D/3D video signal and a 2D/3D audio signal from the 3D video decoding module 230 and the 3D audio decoding module 240, and outputs an AV scene such as a 2D/3D video scene and a 2D/3D audio composing a scene according to BIFS data, which are received from the OD/BIFS decoder 222 of the system analyzing module 220, to the renderer 253.
• the renderer 253 receives and outputs an AV scene such as a 2D/3D video scene and a 2D/3D audio from the scene forming unit 251 be rendering the AV scene.
• the present invention can be embodied as a program and stored in a computer-readable recording medium, such as CD-ROM, RAM, ROM, a floppy disk, a hard disk and a magneto-optical disk. Since the process can be easily implemented by those skilled in the art, further description will not be provided herein.
• the present invention can provide a two dimensional audio/video (2D AV) or a more realistic three dimensional (3D) AV service to a user while maintaining comparability with a conventional digital multimedia broadcasting (DMB) system.
• 2D AV two dimensional audio/video
• 3D three dimensional
• the present invention has an additional effect that the present invention can economically realize a system for providing a 3D AV service by using the conventional DMB system.

Landscapes

• Engineering & Computer Science (AREA)
• Signal Processing (AREA)
• Multimedia (AREA)
• Computer Networks & Wireless Communication (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=35839476

Family Applications (1)

Country Status (4)

Families Citing this family (30)

Citations (7)

Family Cites Families (7)

• 2004
  • 2004-08-09 KR KR1020040062418A patent/KR100658222B1/ko not_active IP Right Cessation
  • 2004-12-30 CN CN2004800437751A patent/CN1998233B/zh not_active Expired - Fee Related
  • 2004-12-30 WO PCT/KR2004/003540 patent/WO2006016735A1/en active Application Filing
  • 2004-12-30 EP EP04808668A patent/EP1776832A4/de not_active Ceased

Patent Citations (7)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events