CN1998233B

CN1998233B - 3-dimensional digital multimedia broadcasting system

Info

Publication number: CN1998233B
Application number: CN2004800437751A
Authority: CN
Inventors: 尹国镇; 曹叔嬉; 徐廷一; 朴基尹; 严基纹; 张恩荣; 姜京玉; 安忠铉; 李寿寅
Original assignee: Electronics and Telecommunications Research Institute ETRI
Current assignee: Electronics and Telecommunications Research Institute ETRI
Priority date: 2004-08-09
Filing date: 2004-12-30
Publication date: 2011-12-07
Anticipated expiration: 2024-12-30
Also published as: WO2006016735A1; EP1776832A4; EP1776832A1; CN1998233A; KR100658222B1; KR20060013818A

Abstract

Provide is a 3-dimensional Digital Multimedia Broadcasting (3D DMB) system. The present invention can provide a 3D DMB system providing a more realistic 3D Audio/Video (AV) service to a user, while maintaining comparability with a conventional DMB system, by processing a binocular 3D image and 3D sound based on a conventional system structure. The 3D DMB system includes: a video encoder for receiving and encoding a video signal of a binocular 3D image; an audio encoder for receiving and encoding an audio signal of a 3D sound; a packetizor for packetizing the encoded video and audio signals into an Sync Layer (SL) packet; and a multiplexer for transforming and multiplexing the SL packet packetized by the packetizer. A 3D DMB receiving system includes: a demultiplexer for demultiplexing a stream including binocular 3D image information and 3D sound information; a depacketizer for depacketizing the SL packet and outputting an encoding stream of an 3D video signal and a 3D sound audio signal; a video decoder for decoding a encoding stream for a video signal of the binocular 3D image; an audio decoder for decoding a encoding stream for the 3D sound audio signal; and a scene restorer for forming and outputting a 2D AV or 3D AV scene by using a video/3D additional video and an audio/3D additional audio signal which are received from the video decoder and the audio decoder.

Description

3-dimensional digital multimedia broadcasting system

Technical field

The present invention relates to a kind of 3-dimensional digital multimedia and broadcast (3-dimensional Digital MultimediaBroadcasting, 3D DMB) system; Relating to 3D DMB particularly sends and receiving system, be used for sending 3D AV by coding 3D audio/video (AV), and provide service for 3DAV to the user by the signal decoding that will be received, so that in the compatibility that keeps with traditional DMB system, provide stereophonic broadcasting more true to nature to the user.

Background technology

Because computer and development of Communication Technique have changed the field of the communication between the people significantly, and replace traditional one directional service and use bi-directional service energetically by various multimedia various data being connected to mobile technology such as DMB (DMB).

The DMB system is used at any time providing the broadcast system of the various multimedia services such as video, audio frequency and data and the service of having prepared the DMB system in the world in Korea S for the first time with any place to the mobile subscriber.

DMB system as transmitting system uses digital audio broadcasting (DAB) to be used as transmitting system, uses MPEG-4 to be used as media processing method, and uses MPEG-4 and MPEG-2 system to be used as being used for standard multiplexing, synchronous and the transmission media data.Can come to provide the Voice ﹠ Video service of high-quality CD level by using the DMB system to the user.

Simultaneously, remarkably researched and developed the technology that is used to handle binocular (binocular) 3D moving image, and attempted integrated 3D moving image and digital broadcasting subsequently with left and right sides image.

In handling the 3D moving image in data volume, have many difficulties synchronously and on the system complexity, these are different with 2 traditional moving images, and, the user in mobile system is not provided for handling the service by the real picture image of camera input, because generally focus is concentrated in computer graphical (CG) processing.

And because the bamboo telegraph of multi-channel sound audio frequency and 3D audio frequency obtain the development with recovery technology, the user increases day by day to the stereosonic demand in the digital broadcasting.

As mentioned above, owing to the interest and the requirement that increase day by day, in the various applications such as physical culture relay, advertisement, education, medical services and recreation, use 3D AV for 3D AV.But, because the focus of DMB service concentrates on the 2DAV, so the realistic and low-down problem of 3D effect of DMB service.

Need to realize new MPEG-4 system information and structure, be used for coding/decoding 3D AV keep simultaneously with the method for the compatibility of traditional DMB system, be used for according to the user the selection of receiving terminal 3D AV is shown as the method for 2D AV or 3D AV and be used to provide 2D or 3D AV service to express the system of 3D AV based on traditional DMB system.

Summary of the invention

Technical problem

Therefore, the purpose of this invention is to provide a kind of three-dimensional (3D) digital multimedia broadcasting system (DMB), be used to provide 3D audio/video (AV) service, described system keeps compatibility with traditional DMB system by using traditional system configuration to handle the 3D moving image of binocular and 3D sound, and provides 3DAV service more true to nature to the user.

To understand other purposes of the present invention and advantage by following explanation, and by as described below according to embodiments of the invention, other purposes of the present invention and advantage will become clearer.Be clear that equally, can easily embody objects and advantages of the present invention by the means of definition in claim and combination thereof.

Technical scheme

According to an aspect of the present invention, provide a kind of 3-dimensional digital multimedia broadcasting (3D DMB) transmitting system, having comprised: video coding apparatus is used to receive and the vision signal of the binocular 3D rendering of encoding; Audio coder is used to receive and the audio signal of the 3D sound of encoding; Object Descriptor/binary format scene generation device is used for Object Descriptor and binary format about binocular 3-D view or three dimensional sound generation scene encoding stream; The synchronization layer packetization device is used for the stream packets from described video coding apparatus, audio coding apparatus and Object Descriptor/binary format scene generation device input is turned to synchronization layer packets; And multiplexer, be used for conversion and multiplexing layer, sync (SL) grouping by described synchronization layer packetization device packetizing, and export multiplexing SL grouping, wherein, described Object Descriptor individually is used for the three-dimensional supplemental video signal about vision signal/3 D video, perhaps, express this 3 D video or three-dimensional audio about the three-dimensional supplemental audio signal of audio signal/three-dimensional audio; Described Object Descriptor comprises from the possession at as the left side of vision signal or two ES_Descriptor of the described three-dimensional supplemental video signal of right video and the right side or left video, to express this 3 D video; And described Object Descriptor comprises low two ES_Descriptor to audio signal and three-dimensional supplemental audio signal of Object Descriptor of pin from the possession, to express this three-dimensional audio.

According to another aspect of the present invention, provide a kind of 3-dimensional digital multimedia broadcasting (3D DMB) receiving system, having comprised: the demultiplexing device is used for comprising that by demultiplexing the stream of binocular 3D rendering information and 3D acoustic information exports the SL grouping; The de-packetization device is used for the described SL grouping of de-packetization, and the encoding stream of output binocular 3D vision signal and 3D wave audio signal; Video decoder is used for about described binocular 3D vision signal decoding and coding stream; Audio decoding apparatus is used for coming decoding and coding stream about described 3D wave audio signal; Object Descriptor/binary format scene decoding device, the described synchronization layer packets that is used to decode, and output binary format scene and Object Descriptor; And scene recovery device, be used for by using from the video/3D supplemental video signal and the audio frequency/3D supplemental audio signal of described video decoder and audio decoding apparatus input, form and export 2D audio/video (AV) or 3D AV scene, wherein, described Object Descriptor individually is used for the three-dimensional supplemental video signal about vision signal/3 D video, perhaps, express this 3 D video or three-dimensional audio about the three-dimensional supplemental audio signal of audio signal/three-dimensional audio; Described Object Descriptor comprises from the possession at as the left side of vision signal or two ES_Descriptor of the three-dimensional supplemental video signal of right video and the described right side or left video, to express this 3 D video; And described Object Descriptor comprises two ES_Descriptor at an Object Descriptor bass signal and three-dimensional supplemental audio signal from the possession, to express this three-dimensional audio.

Description of drawings

By below in conjunction with the description of drawings preferred embodiment, above-mentioned and other purposes of the present invention and feature will become clear, wherein:

Fig. 1 is the block diagram that 3-dimensional digital multimedia broadcasting (3D DMB) transmitting system according to an embodiment of the invention is shown;

Fig. 2 is the block diagram of the video encoding module of diagram Fig. 1;

Fig. 3 is the block diagram of the 3D additional video coding module of diagram Fig. 1;

Fig. 4 is the block diagram of the audio coding module of diagram Fig. 1;

Fig. 5 is the 3D supplemental audio coding module of Fig. 1;

Fig. 6 is the block diagram of the additional audio pre-processor of diagram Fig. 5;

Fig. 7 is the block diagram of the system coding module of diagram Fig. 1;

Fig. 8 illustrates Object Descriptor according to an embodiment of the invention (object descriptor, the block diagram of data structure OD);

Fig. 9 is the block diagram of the M4 on M2 (M4over M2) module that is illustrated in Fig. 1;

Figure 10 is the block diagram that 3D DMB receiving system according to an embodiment of the invention is shown;

Figure 11 is the block diagram of the M2 on M4 (M2over Me) module that is illustrated in Figure 10;

Figure 12 is the block diagram of the Systems Analysis Module of diagram Figure 10;

Figure 13 is the block diagram of the 3D video decode module of diagram Figure 10;

Figure 14 is the block diagram of the 3D audio decoder module of Figure 10;

Figure 15 is the block diagram of the 3D audio post-processor of diagram Figure 14;

Figure 16 is the block diagram of the scene generation module of diagram Figure 10.

Embodiment

By referring to accompanying drawing embodiment being described, other purposes of the present invention and advantage will become clear.Therefore, those skilled in the art in the invention can easily specialize technological thought of the present invention and scope.In addition, then do not provide detailed description by the point fuzziness of wanting of the present invention if think that detailed description about prior art may make at this.And, in digital multimedia transmission of the present invention and receiving system, can can be of value at the authority file of ultrashort wave digital broadcasting video transmission/reception coupling in the scope of function that each element be described and operation, comprise the authority file that the ultrashort wave digital broadcasting video transmission/reception of telecommunication technology federation is mated in this manual.Below describe the preferred embodiments of the present invention in detail referring to accompanying drawing.

Fig. 1 is the block diagram of 3-dimensional digital multimedia broadcasting (3D DMB) transmitting system according to an embodiment of the invention.

3D DMB transmitting system of the present invention is such system, it be used to receive audio/video (AV) data be the additional AV data of vision signal and audio signal and 3D be 3D supplemental video signal and 3D supplemental audio signal, based on MPEG-4 encode these AV data and the additional AV data of 3D and export multiplexing mpeg 2 transport stream (TS).

As shown in fig. 1, described 3D DMB transmitting system comprises video encoding module 110,3D additional video coding module 130,

audio coding module

120,3D supplemental audio coding module 140, system coding module 150 and the M4 module 160 on M2.

At this, video encoding module 110 is to be used for one of the left image of binocular 3D moving image and right image are encoded to MPEG-4 part 10 advanced video codings (Advanced Video Coding, AVC) module of baseline class (baseline profile) specification.

Audio coding module 120 is that the audio-frequency signal coding that is used for sending from the outside is MPEG-4 bit sliced arithmetic coding (Bit Sliced Arithmetic Coding, the BSAC) module of specification.

3D supplemental audio coding module 140 is that being used for the 3D supplemental audio signal encoding from the outside input is the module of MPEG-4 bit BSAC specification.

System coding module 150 produces, encoded MPEG-4 initial object descriptor (Initial ObjectDescriptor, IOD)/OD/ binary format scene (Binary Format for Scene, BIFS) data, and (Elementary Stream, ES) Object Descriptor of Chan Shenging (OD)/binary format scene (BIFS) stream and the packet that receives from the outside turn to layer, sync (SL) and divide into groups by using basic stream from described 4 coding module 110-140 output.

M4 module 160 on M2 receives the SL grouped data, and the grouping of SL grouped data is turned to MPEG-2TS, packetized MPEG-2TS is multiplexed with a MPEG-2TS, and exports described MPEG-2TS.

Fig. 2 is the block diagram of the video encoding module of diagram Fig. 1.

As shown in Figure 2, video encoding module 110 comprises video pre-processor (pre-processor) 111 and video encoder 113.Video pre-processor 111 receives a left side or the right vision signal about binocular 3D moving image from the outside, and with the size conversion of described image is 1/4th Video Graphics Arrays (QuaterVideo Graphic Array, QVGA) 320 * 240 or public intermediate form (Common IntermediateFormat, CIF) 352 * 288 size.Video encoder 113 output videos flow (ES) substantially, and it is to become so that MPEG-4 part 10 advanced video codings (AVC) baseline class specification and coding parameter---it is that bit rate obtains by the video signal coding that size is converted in additional video pre-processor 111.At this, video encoding module 110 can receive and handle general 2D motion picture video signal.

Fig. 3 is the block diagram of the 3D additional video coding module of diagram Fig. 1.

As shown in Figure 3,3D additional video coding module 130 comprises additional video pre-processor 131 and additional video encoder 133.Additional video pre-processor 131 receives a left side or the right vision signal about binocular 3D moving image from the outside, and with the size conversion of described image is 160 * 240/320 * 120 or 176 * 288/352 * 144, and it is reduced on height or width is half of described QVGA/CIF size.Additional video encoder 133 output 3D additional video ES, it is that MPEG-4 part 10AVC baseline class specification and coding parameter---it is that bit rate obtains by the video signal coding after the conversion in additional video pre-processor 131 is become.

Fig. 4 is the block diagram of the audio coding module of diagram Fig. 1.

As shown in Figure 4, audio coding module 120 comprises audio pre-processor 121 and audio coder 123.The resolution of the audio signal that audio pre-processor 121 conversion receive from the outside and sample frequency are to satisfy the MPEG-4BSAC standard.Audio coder 123 be received in the audio signal after the conversion in the audio pre-processor 121 and the dual track that in 3D supplemental audio coding module 140, produces under mix (downmix) audio signal, and also produce audio ES, described audio ES is by based on coding parameter---it is a bit rate---and be encoded to the MPEG-4BSAC specification.

Fig. 5 is the 3D supplemental audio coding module of Fig. 1 according to an embodiment of the invention.

As shown in Figure 5,3D supplemental audio coding module 140 comprises additional audio pre-processor 141 and additional audio encoder 143.Additional audio pre-processor 141 is mixed (downmix) signal and 3 sound channel 3D supplemental audio signals by using the 3D supplemental audio signal that receives from the outside to produce and export under the dual track, wherein, mix the audio coder 123 that (downmix) signal is output to MPEG-4 audio coding module 120 under the described dual track.Additional audio encoder 143 is by being that the MPEG-4BSAC specification is exported 3D supplemental audio ES with described 3 sound channel 3D supplemental audio signal encodings.At this, described 3D supplemental audio signal can be various multichannel sterego signals, such as 5 channel stereo signal or 5.1 channel stereo signal, described 5 channel stereo signal comprise left loudspeaker signal L, right front speaker signal R, preceding middle loudspeaker signal C, left rear speaker signal LS and right rear loudspeakers signal RS, in described 5.1 channel stereo signal, added inferior bass (Sub-Woofer) signal SW to described 5 signals.

Fig. 6 is the block diagram of the additional audio pre-processor of diagram Fig. 5.

As shown in Figure 6, supplemental audio pre-treatment piece 141 comprises front processor 145 and channel-mixer 147.Front processor 145 receives the 3D supplemental audio signal as various channel stereo signal from the outside, handle the characteristics of signals such as sample frequency and quantization bit number, and produces 5 channel stereo signal of wherein carrying out low-pass filtering.Channel-mixer 147 produces mixed signal LO and R0 and 3 sound channel supplemental audio signal T, Q1 and Q2 under the dual track by using the mixed method by ITU-R BS.775-1 normalized definition from described 5 channel stereo signal.

At this, mixed signal LO and R0 are a left side/right audio signals that uses when stereo when playing under the described dual track, and described 3 sound channel supplemental audio signal T, Q1 and Q2 are used to recombinate the signal of primary signal.And when the 3D supplemental audio signal that is input to front processor 145 was 5 channel stereo signal, front processor 145 can need not to export the signal of being imported under the situation of any special processing.

Fig. 7 is the block diagram of the system coding module of diagram Fig. 1.

As shown in Figure 7, system coding module 150 comprises SL packetizer (packetizer) 151, OD/BIFS generator 153 and initial object descriptor (IOD) generator 155.

At this, OD/BIFS generator 153 produces and exports OD/BIFS ES by using the OD/BIFS text data that receives from the outside.By individually using an OD to express 3D AV about video/3D additional video and audio frequency/3D supplemental audio.

And, SL packetizer 151 is from above-mentioned 4 coding module receiver, video ES, audio ES, 3D additional video ES, 3D supplemental audio ES and OD/BIFS ES, and by according to Korea S's Mobile Multimedia Broadcasting standard wherein each grouping turn to SL and divide into groups to export them.

IOD generator 155 produces and exports IODES by using the IOD text data that sends from the outside.

Fig. 8 is the block diagram that the data structure of Object Descriptor according to an embodiment of the invention (OD) is shown.

In video, be included in from the possession the OD at two ES_Descriptor (ES descriptor) of a left side or the 3D additional video of right video and the right side or left video, in the OD/BIFS structure compatible that keeps with traditional DMB system, to express 3D AV.And, in audio frequency, at an OD bass and and two ES_Descriptor of 3D supplemental audio have subordinate (subordinate) relation.

Can express described subordinate relation by Stream DependenceFlag and the depends On_ES_ID that uses MPEG-4OD.

And the ObjectTypeIndication of 3D additional video ES is set to User Private (user's special use), and stream type (stream type) is set to Visual (visual), keeps and traditional DMB system compatible simultaneously so that 3D AV service to be provided.The ObjectTypeIndication of 3D supplemental audio ES is set to User Private (user's special use), and stream type (stream type) is set to Audio (audio frequency).Other video and audio ES are set according to traditional DMB system specifications.

Tradition DMB system can not come the information about the additional AV stream of 3D identification ObjectTypeIndication by this, and can not receive the additional AV encoding stream of relevant 3D.

As a result, because traditional DMB system can discern only general 2D video and audio frequency, and receive only relevant encoding stream, therefore provide backward compatibility to the user, it can be received as the 2D moving image about 3DAV.

Fig. 9 is the block diagram of the M4 module on M2 of diagram Fig. 1.

As shown in Figure 9, M4 module 160 on M2 comprises that (PacketizedElemenary Stream, PES) packetizer 161, PES partly arrive TS packetizer 167 and 14496 parts to TS packetizer 168 to TS packetizer 162, multiplexer 163, switch 164,14496 part packetizer 165, PSI generator 166, program specific information PSI for the basic stream of packetizing.

Switch 164 receives OD/BIFS SL grouping from system coding module 150, and comes to PES packetizer 161 or the 165 output OD/BIFSSL groupings of 14496 part packetizer according to the coded message of SL grouping.

The video SL grouping that receives from system coding module 150, audio frequency SL grouping, 3D additional video SL grouping, 3D supplemental audio SL grouping and divided into groups respectively to turn to PES by PES packetizer 161 from the OD/BIFS SL grouping of switch 164 inputs, and output to PES to TS packetizer 162.

Turned to 14496 parts from the OD/BIFS SL grouping of switch 164 inputs by 165 groupings of 14496 part packetizer, and output to TS packetizer 168.

The IOD information that PSI generator 166 receives from system coding module 150 by use produces and comprises Program Association Table (Program Association Table, PMT) partly (PA_section) and Program Map Table (Program Map Table, PMT) partly the PSI of (PM_section), and partly arrive the described PSI of TS packetizer 167 outputs to PSI.

The PES grouping of being imported is turned to MPEG-2TS and outputs to multiplexer 163 to 162 groupings of TS packetizer by PES.

Ground 14496 parts of importing are turned to MPEG-2TS by 14496 parts to 168 groupings of TS packetizer, and output to multiplexer 163.

Described PSI information is partly turned to MPEG-2TS to 167 inputs of TS packetizer and grouping by PSI, and outputs to multiplexer 163.

The MPEG-2TS that the is imported device 163 that is re-used is multiplexed with a transport stream, and multiplexing MPEG-2TS is output.

Figure 10 is the block diagram that 3D DMB receiving system according to an embodiment of the invention is shown.

3D DMB receiving system of the present invention is the multiplexing MPEG-2TS that is used for demultiplexing and decodes and import from the outside, and shows the system of 2D AV or 3D AV to the user.

As shown in Figure 10, described 3D DMB receiving system is included in M2 module 210,

Systems Analysis Module

220,3D

video decode module

230,3D audio decoder module 240 and the scene generation module 250 on the M4.

At this, the multiplexing MPEG-2TS that the M2 module 210 on M4 is imported from the outside by demultiplexing and de-packetization comes output video SL grouping, audio frequency SL grouping, 3D additional video SL grouping, 3D supplemental audio SL grouping, OD/BIFS SL to divide into groups and the IOD data.

Systems Analysis Module 220 de-packetizations comprise the SL that the is received grouping of video SL grouping, audio frequency SL grouping, 3D additional video SL grouping, 3D supplemental audio SL grouping and OD/BIFS SL grouping, described SL grouping is output as the encoding stream that comprises video ES, audio ES, 3D additional video ES and 3D supplemental audio ES, and exports IOD data and OD/BIFS data by decoding IOD data and OD/BIFS data.

3D video decode module 230 is output as 2D video scene or 3D video scene by this video ES and the 3D additional video coding ES that decodes with video ES and 3D additional video coding ES.

3D audio decoder module 240 is 2D or 3D audio signal by decode this audio ES and 3D supplemental audio ES with the 3D audio signal output.

Scene generation module 250 is by using from the 2D/3D vision signal and the 2D/3D audio signal of 3D

video decode module

230 and 240 receptions of 3D audio decoder module to form and export scene according to the definition of BIFS.

Figure 11 is the block diagram of the M2 module on M4 of diagram Figure 10.

As shown in Figure 11, the M2 module 210 on M4 comprise demultiplexer 211, TS to PES de-packetizer 212, TS to 14496 part de-packetizer 213, TS to PSI part de-packetizer 214, PES de-packetizer 215,14496 partial analysis devices 216 and PSI partial analysis device 217.

Demultiplexer 211 individually demultiplexing and is output as single MPEG-2TS about video ES, audio ES, 3D additional video ES, 3D supplemental audio ES, OD/BIFS, 14496 parts and PSI with the MPEG-2TS of demultiplexing from the MPEG-2TS that is re-used of outside input.

The MPEG-2TS of TS 211 receiver, videos, audio frequency, 3D additional video, 3D supplemental audio and OD/BIFS to PES de-packetizer 212 from demultiplexer, and by being that PES divides into groups described MPEG-2TS is outputed to PES de-packetizer 215 with the MPEG-2TS de-packetization.

TS receives the MPEG-2TS of 14496 parts to 14496 part de-packetizer 213, and by with the MPEG-2TS de-packetization being 14496 parts and to the described MPEG-2TS of 14496 partial analysis devices, 216 outputs.

TS receives the MPEG-2TS of PSI to PSI part de-packetizer 214, and by being that PMP partly comes to export described MPEG-2TS to PSI partial analysis device 217 with described MPEG-2TS de-packetization.

PES de-packetizer 215 is 212 each PES grouping that receives about video, audio frequency, 3D additional video, 3D supplemental audio and OD/BIFS from TS to the PES de-packetizer, and by being that SL divides into groups to come output video SL grouping, audio frequency SL grouping, 3D additional video SL grouping, 3D supplemental audio SL grouping and OD/BIFS SL grouping with described PES grouping de-packetization.

14496 partial analysis devices 216 receive 14496 parts from TS to 14496 part de-packetizer 213, and divide into groups to export OD/BIFS SL grouping by extracting OD/BIFS SL.

PSI partial analysis device 217 receives PAT part and PMT part, and exports the IOD data by extracting the IOD data.

Figure 12 is the block diagram of the Systems Analysis Module of diagram Figure 10.

As shown in Figure 12, Systems Analysis Module 220 comprises SL de-packetizer 221, OD/BIFS decoder 222 and IOD decoder 223.

SL de-packetizer 221 is from the 210 receiver, video SL of the M2 module on M4 grouping, audio frequency SL grouping, 3D additional video SL grouping, 3D supplemental audio SL grouping and OD/BIFS SL grouping, with their each de-packetization is encoding stream, and output video ES and output 3D additional video ES are to 3D video decode module 230, output audio ES and 3D supplemental audio ES are to 3D audio decoder module 240, and output OD/BIFS data are to OD/BIFS decoder 222.

The OD/BIFS coded data that OD/BIFS decoder 222 decoding sends from SL de-packetizer 221, and to the BIFS information of scene generation module 250 output decoders, wherein, the OD information of being decoded is used for each media decoder of initialization.

IOD decoder 223 is exported the IOD coded data by receiving the decode the IOD coded data, and the IOD data of decoding are used to extract the OD/BIFS data.

Figure 13 is the block diagram of the 3D video decode module of diagram Figure 10.

As shown in Figure 13,3D video decode module 230 comprises Video Decoder 231,

additional video decoder

233 and 3D video post-processor 235.

Video Decoder 231 receives the video ES of the MPEG-4 part 10AVC specification of 320 * 240/352 * 288 image sizes from the SL de-packetizer 221 of Systems Analysis Module 220, and comes to the described video ES of 3D video post-processor 235 outputs by the described video ES that decodes.

Additional video decoder 233 receives the 3D additional video ES of the MPEG-4 part 10AVC specification of 160 * 240/320 * 120 or 176 * 288/352 * 144 image sizes from the SL de-packetizer 221 of Systems Analysis Module 220, and comes to 3D video post-processor 235 output 3D additional video ES by the described 3D additional video ES that decodes.

3D video post-processor 235 receives the vision signal and the 3D supplemental video signal of decoding from Video Decoder 231 and additional video decoder 233 respectively, and when being 3D video recovery pattern, the video recovery pattern information from user input vision signal and 3D supplemental video signal are output as the 3D vision signal of QUGA320 * 240 or CIF352 * 288 by coming composite signal according to the field, perhaps when the video recovery pattern information is 2D video recovery pattern, only export the 2D vision signal of 320 * 240/352 * 288 image sizes, and ignore the 3D supplemental video signal that is received.The user can select the display packing of expectation to appreciate 3D AV by this.

Figure 14 is the block diagram of the 3D audio decoder module of Figure 10.

As shown in Figure 14,3D audio decoder module 240 comprises audio decoder 241, supplemental audio decoder 243 and 3D audio post-processor 245.

Audio decoder 241 receives the audio ES of MPEG-4BSAC specifications from the SL de-packetizer 221 of Systems Analysis Module 220, and by decoded audio ES and to the described audio ES of 3D audio post-processor 245 outputs.

Supplemental audio decoder 243 receives the 3D supplemental audio ES of MPEG-4BSAC specifications from the SL de-packetizer 221 of Systems Analysis Module 220, and comes to the described 3D supplemental audio ES of 3D audio post-processor 245 outputs by the described 3D supplemental audio ES that decodes.

3D audio post-processor 245 receives the audio signal and the 3D supplemental audio signal of decoding from audio decoder 241 and supplemental audio decoder 243, produces and

export

2D or 3D audio signal by using from the audio frequency recovering information of user's input.

Figure 15 is the block diagram of the 3D audio post-processor of diagram Figure 14.

As shown in Figure 15,3D audio post-processor 245 comprise 3D/3D switch 246, remove matrixing (de-matrixer) device 247 and virtual 3D synthesizer 248.

When 2D/3D switch 246 is 3D audio frequency recovery pattern when the audio frequency recovery pattern information that receives from the user, the 3D audio signal L0, R0, T, Q1 and the Q2 that export 5 sound channels by the audio signal and the 3D supplemental audio signal of synthetic decoding, perhaps when audio frequency recovery pattern information is 2D audio frequency recovery pattern, only export 3D audio signal L0 and R0, and ignore the 3D supplemental audio signal that is transfused to.

Go matrixing device 247 received audio signals and 3D audio signal, and by using the ITU-RBS.775-1 standard to export 5 channel audio signal L, R, C, LS and RS.

Virtual 3D synthesizer 248 receives and exports described 5 channel audio signal by the 3D audio signal that virtual acoustic signal L and the R that described 5 channel audio signal is transformed to by dual track forms.

As shown in figure 16, scene generation module 250 comprises that scene forms unit 251 and renderer (renderer) 253.

Scene forms the AV data that unit 251 receives such as 2D/3D vision signal and 2D/3D audio signals from 3D

video decode module

230 and 3D audio decoder module 240, and to the BIFS data that renderer 253 outputs receive according to the OD/BIFS decoder 222 from Systems Analysis Module 220 constitute scene, such as the AV scene of 2D/3D video scene and 2D/3D audio frequency.

Renderer 253 receives and exports the AV scene such as 2D/3D video scene and 2D/3D audio frequency that forms unit 251 from scene by presenting described AV scene.

Therefore, might realize a kind of system, it can provide 3D AV content service to the user in maintenance and traditional DMB system compatible.

As described in detail, the present invention can be embodied as program, and is stored in the computer readable recording medium storing program for performing, such as CD-ROM, RAM, ROM, floppy disk, hard disk and magneto optical disk.Because can easily realize described processing, therefore do not further specify at this by those skilled in the art.

The present invention can provide two-dimentional audio/video (2D AV) or three-dimensional (3D) AV service more true to nature to the user in maintenance and conventional digital multimedia broadcasting (DMB) system compatible.

And the present invention has additional effect: the present invention can realize a kind of system of the 3D of providing AV service by using traditional DMB systematic economy ground.

Though the present invention has been described referring to the preferred embodiment of some, very clear for those skilled in the art, can carry out various changes and modification not breaking away under the situation of scope of the present invention as defined by the appended claims.

Claims

1. 3-dimensional digital multimedia broadcast transmission system comprises:

Video coding apparatus is used to receive and the vision signal of the binocular 3-D view of encoding;

Audio coding apparatus is used to receive and the audio signal of the three dimensional sound of encoding;

Object Descriptor/binary format scene generation device is used for Object Descriptor and binary format about binocular 3-D view or three dimensional sound generation scene encoding stream;

The synchronization layer packetization device is used for the stream packets from described video coding apparatus, audio coding apparatus and Object Descriptor/binary format scene generation device input is turned to synchronization layer packets; And

Multiplexer is used for conversion and multiplexing synchronization layer packets by described synchronization layer packetization device packetizing, and exports multiplexing synchronization layer packets,

Wherein, described Object Descriptor individually is used for the three-dimensional supplemental video signal about vision signal/3 D video, perhaps about the three-dimensional supplemental audio signal of audio signal/three-dimensional audio, expresses this 3 D video or three-dimensional audio;

Described Object Descriptor comprises from the possession at as the left side of vision signal or two ES_Descriptor of the described three-dimensional supplemental video signal of right video and the right side or left video, to express this 3 D video; And

Described Object Descriptor comprises two ES_Descriptor at an Object Descriptor bass signal and three-dimensional supplemental audio signal from the possession, to express this three-dimensional audio.

2. according to the transmitting system of claim 1, wherein, described video coding apparatus comprises:

First video coding apparatus, a left side or the right vision signal of the binocular 3-D view that is used to encode are as the vision signal of 3 D video; And

Second video coding apparatus, be used for being coded among a described left side and the right vision signal at the uncoded signal of this first video coding apparatus, as the three-dimensional supplemental video signal of 3 D video.

3. according to the transmitting system of claim 1, wherein, described audio coding apparatus comprises:

First audio coding apparatus is used for encoding mono audio signal or stereo audio signal, as the audio signal of three-dimensional audio; And

Second audio coding apparatus, the 3 sound channel supplemental audio signals that are used to encode are as the three-dimensional supplemental audio signal of three-dimensional audio.

4. according to the transmitting system of claim 2, wherein, described first video coding apparatus comprises:

The first video pretreating device is used for the image size of the vision signal that conversion imports; And

First video coding apparatus is used for being coded in the signal after this first video pretreating device conversion.

5. according to the transmitting system of claim 4, wherein, described second video coding apparatus comprises:

The second video pretreating device, the size conversion that is used for the vision signal that will be imported is for being half image size of the size after this first video pretreating device conversion on height or width; And

Second video coding apparatus is used for being coded in the signal after this second video pretreating device conversion.

6. according to the transmitting system of claim 1, wherein, described Object Descriptor comprises object-type information, and being used for provides back compatible about described three-dimensional supplemental video signal or described three-dimensional supplemental audio signal.

7. according to the transmitting system of claim 1, wherein said ES_Descriptor will be established as Visual about the Stream Type of described three-dimensional supplemental video signal, ObjectTypeIndication is established as User private, to be established as Audio about the Stream Type of described three-dimensional supplemental audio signal, and ObjectTypeIndication is established as User private, so that provide back compatible about described three-dimensional supplemental video signal or described three-dimensional supplemental audio signal.

8. 3-dimensional digital multimedia broadcast receiver system comprises:

The demultiplexing device is used for comprising that by demultiplexing the stream of binocular three-dimensional image information and three dimensional sound information exports synchronization layer packets;

The de-packetization device is used for the described synchronization layer packets of de-packetization, and the encoding stream of output binocular 3 D video signal and three dimensional sound audio signal;

Video decoder is used for coming decoding and coding stream about described binocular 3 D video signal;

Audio decoding apparatus is used for coming decoding and coding stream about described three dimensional sound audio signal;

Object Descriptor/binary format scene decoding device, the described synchronization layer packets that is used to decode, and output binary format scene and Object Descriptor; And

The scene recovery device is used for forming and export two-dimentional audio or video or three-dimensional audio or video scene by using from video or three-dimensional supplemental video signal and the audio frequency or the three-dimensional supplemental audio signal of described video decoder and audio decoding apparatus input,

Described Object Descriptor comprises from the possession at as the left side of vision signal or two ES_Descriptor of the three-dimensional supplemental video signal of right video and the described right side or left video, to express this 3 D video; And

9. receiving system according to Claim 8, wherein, described video decoder comprises:

First video decoder is used for by come a left side or the right picture decoding video encode stream about described binocular 3 D video signal from described de-packetization device receiver, video encoding stream, as the vision signal of 3 D video;

Second video decoder, be used for by coming from described de-packetization device receiver, video encoding stream and the decoded video encoding stream, as the three-dimensional supplemental video signal of 3 D video about signal among this left side and right video, that be not transfused to first video decoder; And

The video after-treatment device is used for according to the video recovery pattern information from the outside input vision signal by this first video decoder and the decoding of this second video decoder being output as two-dimensional video signal or 3 D video signal.

10. according to the receiving system of claim 9, wherein, be half at height or width from the image size of the video encode stream of this first video decoder reception from the image size of the video encode stream of described second video decoder input.

11. receiving system according to Claim 8, wherein, described audio decoding apparatus comprises:

First audio decoding apparatus is used for coming decoding mono encoded audio stream or stereo audio coding stream by receive monophonic audio encoding stream or stereo audio coding stream from described de-packetization device, as the audio signal of three-dimensional audio;

Second audio decoding apparatus is used for by receive 3 sound channel supplemental audio encoding streams, the 3 sound channel supplemental audio encoding streams of decoding from described de-packetization device, as the three-dimensional supplemental audio signal of three-dimensional audio; And

The audio frequency after-treatment device is used for recovering pattern information according to the audio frequency from the outside input, will be two-dimentional audio signal or three-dimensional audio signal by the audio signal output of this first audio decoding apparatus and the decoding of this second audio decoding apparatus.