CN100534199C

CN100534199C - Seamless switching between random access units multiplexed in a multi angle view multimedia stream

Info

Publication number: CN100534199C
Application number: CNB2005800181453A
Authority: CN
Inventors: 远间正真; 冈田智之; 角野真也
Original assignee: Matsushita Electric Industrial Co Ltd
Current assignee: Panasonic Holdings Corp
Priority date: 2004-06-02
Filing date: 2005-06-01
Publication date: 2009-08-26
Anticipated expiration: 2025-06-01
Also published as: US20100021139A1; CN101707703B; CN101707720B; CN101707702A; US8411759B2; CA2555700C; PL2051532T3; TWI324012B; KR20100033443A; ES2342448T3; JP4790776B2; EP2180713A2; CN101594530B; EP2180712A2; TW200952462A; CN101594530A; TWI323602B; US20100046638A1; KR20100035720A; TWI323123B

Abstract

The multiplexing apparatus which does not make a user who watches a moving picture feel uncomfortable, the moving picture including: (i) a coding unit (42) for generating one or more coded streams so that no gap occurs at the connection of predetermined two access units in a process of decoding, in sequence, the two access units in all of the access units included in the coded streams; and (ii) a multiplexing unit (18) for multiplexing, with other information, the coded streams generated in the coding unit (42).

Description

Seamless switching in multi-media stream at multi-angle perspective between the multiplexed random access units

Technical field

The present invention relates to a kind of multiplexed one or more encoding streams that comprise coded picture and out of Memory that are used for, with the multiplex machine that produces multiplexed data with to the Deplexing apparatus of multiplexed data demultiplexing.The present invention be more particularly directed to carry out the multiplex machine and the Deplexing apparatus of the special play-back such as multi-angle broadcast etc.

Background technology

Recently, the multimedia era that sound, picture and other pixel value are integrated in a kind of medium arrives, and also regards the existing information medium of the means of communication of newspaper, magazine, TV, radio and phone and so on as multimedia target.Generally speaking, multimedia is a kind of not only with character, and the form that image, sound, particularly picture are manifested simultaneously.For above-mentioned existing information medium are handled as multimedia, need digitally presents information.

But, utilize above-mentioned existing information medium directly unpractical during digital processing flood tide information, because when the data volume of above-mentioned every kind of information medium is calculated as amount of digital data, the data volume of each character is 1 to 2 byte, and the data volume of per second sound is not less than 64 kilobytes (call voice quality), and the data volume of per second motion picture then is not less than 100 Mbytes (the current TV quality of reception).For example, owing to have the appearance of the Integrated Service Digital Network(ISDN) (ISDN) of the transmission speed of 64kbps to 1.5mbps, the TV phone has dropped into commercial the use, still, when the motion picture of TV video camera uses ISDN, then can not transmit them.

This is the reason that needs compressionism why.For example, International Telecommunications Union-telecommunication standardization portion (ITU-T) is recommended H.261 or motion picture compress technique standard H.263 be used for the TV phone.In addition, utilize the compressionism of Moving Picture Experts Group-1, also make it possible to image information and acoustic information together are stored among the common music CD (CD).

Here, motion picture expert group (MPEG) is the digital compression moving picture signals, and by the standardized international standard of ISO/IEC (International Standards Organization/international project association (InternationalStandardization Organization/Internation Engineering Consortium)).MPEG-1 a kind ofly is compressed to 1.5Mbps with moving picture signals,, is compressed to about centesimal standard that is.In addition, the quality that satisfies Moving Picture Experts Group-1 is the medium level that can realize in the about transfer rate of 1.5Mbps.Therefore, so that satisfy the more needs of high picture quality, and it is compressed to 2 to 15Mbps with moving picture signals with Moving Picture Experts Group-2ization.Current, the working group of standardization MPEG-1 and MPEG-2 (ISO/IEC JTC1/SC29/WG11) will have the more MPEG-4 standardization of high compression rate.MPEG-4 standard (i) has obtained than MPEG-1 and the higher compression ratio of MPEG-2, (ii) allows to encode on an object connects the basis of an object decoding and executable operations and (iii) realized the new functional requirement in the multimedia era.The initial purpose of MPEG-4 is to want standardization to have the coding method of the picture of low bit rate, and still, target has expanded to the coding method of the general purpose of the interlacing scan picture with high bit rate.After this, ISO/IEC and the change of ITU-T common standard as the MPEG-4AVC (advanced video coding) of picture coding method of future generation of picture with high compression rate.Expection is used for optical disc apparatus of future generation or uses in the broadcasting to portable terminal.

Generally speaking, in to motion picture encoding, come the compressed information amount by minimizing time and spatial redundancy.In at the inter-picture prediction coding that reduces time redundancy, by with reference at preceding picture or picture subsequently, connect at one and to carry out locomotion evaluation on one the basis and predictive picture produces, and the difference between predictive picture that obtains and the picture that will encode is encoded." picture (Picture) " used herein is the term of representing a picture.In picture line by line, it represents a frame; In interlaced picture, it represents a frame or one." interlacing scan picture " expression described here is by two frames that the field constitutes with slight time lag.In the Code And Decode of interlaced picture, a frame can be handled as a frame, a frame is handled as two fields, or in frame, on the basis of block-by-block, frame structure is handled or handled as field structure.

Being used to carry out not, the picture of the intraperdiction encoding of any reference picture of reference is called interior coded picture (Intra Coded Picture) (I picture).In addition, the picture that only is used to carry out with reference to the mutual predictive coding of a picture is called prediction encoded picture (Predictive Coded Picture) (P picture).The picture that is used to carry out with reference to the mutual predictive coding of two reference pictures is called Bi-prediction encoded picture (Bi-predictive Coded Picture) (B picture).The B picture can be with reference to being chosen as on the demonstration time preceding and at two pictures of the combination in any of back picture.These two reference pictures can connect on the basis at one and stipulate that piece is the base unit of Code And Decode.The following mutual differentiation of these reference pictures: the reference picture that early illustrates in the coded bit stream is called first reference picture, and other reference picture of explanation is later on called second reference picture.Should be noted that for the I picture of encoding or decode, P picture and B picture these reference pictures must be encoded or decoding.

The motion compensation intraperdiction encoding is used for the coding of P picture and B picture.The motion compensation intraperdiction encoding is a kind of intraperdiction encoding method of using motion compensation.Motion compensation is a kind of amount of exercise (hereinafter referred to as motion vector) of each piece by the estimation picture and considers that by execution the predictive coding of motion vector improves precision of prediction and reduces the method for data volume.For example, the motion vector by the picture that will encode of estimation and by give displacement each predicted value of amount of each motion vector encode with each the prediction surplus between each the current picture that will encode, and reduce data volume.Under situation about making in this way, owing in decoding, need motion vector information, so also will encode, write down or send motion vector.Motion vector is to estimate on a macro block connects the basis of a macro block.More particularly, motion vector be by the fixing macro block that will encode, in the hunting zone mobile reference picture macro block and find that the position of the reference macroblock of close calibrated bolck is estimated.

Figure 1A and 1B are respectively the structure charts of existing MPEG-2 stream.

As shown in Figure 1B, the hierarchical structure that will illustrate below MPEG-2 stream has.Stream is made of picture group (Group of Pictures) (hereinafter referred to as GOP).GOP can be edited motion picture or carries out arbitrary access as the base unit in the encoding process process.GOP is made of I picture, P picture and B picture.Stream, GOP and picture further comprise the synchronizing signal (sync) on the border of indicating member and indicate the stem of corporate data in each unit that each unit is respectively stream, GOP and picture here.

Fig. 2 A and 2B show example how to carry out the inter-picture prediction coding that uses among the MPEG-2 respectively.

The picture that has oblique line among the figure is by the picture of other picture reference.As shown in Fig. 2 A, in the predictive coding in MPEG-2, P picture (P0, P6, P9, P12 and P15) can be only tightens the I picture before nestling or the single picture of P picture with reference to being chosen as in the demonstration time.B picture (B1, B2, B4, B5, B7, B8, B10, B11, B13, B14, B16, B17, B19 and B20) can be with reference to two pictures selecting as the combination of adjacent forward I picture or I picture after P picture and adjacent the leaning on or P picture.In addition, determine the order that picture is placed in stream.A plurality of I pictures and a P picture are according to the order placement that shows the time, and each B places after being close to the I picture that and then the B picture shows afterwards, or and then P picture placement afterwards.As the example of structure of GOP, as shown in Fig. 2 B, the picture from I3 to B14 is formed a single GOP.

Fig. 3 A to 3B shows decoding order respectively, DISPLAY ORDER, and the decode time of the gop structure that uses in MPEG-2 stream and the retardation of the generation between the demonstration time.

At this, MPEG-2 stream has fixing frame frequency, and decodes simultaneously and show the P picture.In MPEG-2 stream, shown in Fig. 3 A and 3B, equal a frame or two fields as retardation maximum from the decode time of GOP top field to the time lag of the time of delay of top field.Hereinafter this retardation is called frame delay, and on basis frame by frame, calculates the length (frame is corresponding to two) of frame delay.Optical disc apparatus such as digital versatile dish (DVD) adopts Moving Picture Experts Group-2, and wherein definition frame postpones to be fixed to 1.Be noted that retardation is variable when for example showing drop-down (pulling down) with the stream of 24Hz coding with 60Hz.Owing to can the situation according to frame frequency code displaying stream be described below according to determining retardation according to the situation of frame frequency code displaying stream.

Fig. 4 is the structure chart of MPEG-4AVC stream.In MPEG-4AVC, there is not notion with the GOP equivalence.But, because can be by in the unit of the specific picture that can not rely on other picture codec, data sementation being constructed the random access units that is equivalent to GOP, below this unit be called RUA (random access units (Random Access Unit)).

Two types I picture is arranged among the MPEG-4AVC.They are instantaneous decoder refresh (IDR) picture and another kind of type.The IDR picture is can be with reference to the picture that is placed on by decoding order before the IDR picture, can be to be placed on the I picture of all picture codecs after the IDR by decoding order.The IDR picture is corresponding to the picture of I foremost of the closed GOP of MPEG-2.At the I picture is not under the situation of IDR picture, and the picture afterwards that is positioned at this I picture by decoding order can be with reference to the picture that is positioned at this I picture front by decoding order.In addition, by the I picture that is not the IDR picture being placed on the predict of the picture among random access units RAU top and the restriction random access units RAU, can form the structure that resembles open GOP among the MPEG-2.

Fig. 5 is an example of the predict of picture in the MPEG-4AVC stream.Because MPEG-4AVC allows predict flexibly, for example, picture P2 can reference picture P8.In the example of Fig. 5, owing to after picture I8, begin to show and at first picture P2 is decoded that frame delay becomes 2.Because predict is flexibly in this mode, as the situation of MPEG-2, the frame delay maximum is not limited to 1.This means that frame delay can change according to predict.Therefore, can not be fixed as 1 with frame delay is that condition is play.

Packet medium such as DVD have trick play functionality, and for example (i) plays, and promptly play the specific part of same stream selectively or can play not homogeneous turbulence and (ii) multi-angle broadcast continuously, promptly carry out the broadcast that changes stream with different angles.The elementary cell of using this function is GOP among the MPEG-2 and the random access units RAU among the MPEG-4AVC.

Fig. 6 A to 6C shows the example of the change stream that will play among the MPEG-2.Fig. 6 A to 6C shows the GOP that comprises in the

stream

1,2 and 3 respectively.At this, by decoding makes the stream that will play change to stream 2 from flowing 1 to GOP2-1 behind GOP1-1.Because the frame delay amount all is 1 at GOP1-1 and GOP2-1 in the two, it is play with fixing speed, and do not allow to occur the gap when showing.Equally, by behind GOP1-1, GOP3-1 being decoded, can change to stream 3 from stream.

Routinely, proposed to resemble and related to motion picture encoding above-mentioned, multiplexed, the various technology of decoding and demultiplexing.(for example, with reference to Japanese Patent Application Publication No.2003-18549).

Fig. 7 is that expression is used for the flow chart to the operation of moving picture data coding and multiplexed traditional multiplex machine.

At first, in

step

101 and 102, multiplex machine is to one or more stream encryptions.Next,, produce management information, proceed to step 104 then in step 103.Management information comprises the information of the data that the information that is used to be accessed in the stream that step 101 produces and indication will be play when the special play-back such as multi-angle broadcast etc.After this, in step 104 that management information and flow data is multiplexed, to export multiplexing data.

Fig. 8 is the block diagram of the structure of the traditional multiplex machine of expression.

Multiplex machine 800 comprises coding unit 11, memory 12, management information generation unit 13 and multiplexed unit 14.

The moving picture data Vin of 11 pairs of inputs of coding unit encodes, and coded data strIn is stored in the memory 12.

Management information generation unit 13 is read coded data from memory 12, as sense data strOut1, produces management information bank and exports these management information banks to multiplexed unit 14.Be noted that this management information bank does not comprise the information relevant with frame delay.

Multiplexed unit 14 multiplexed following data: (i) management information bank, the (ii) sense data strOut2 that has read from memory 12, (iii) additional information adInf, for example, that be provided with by the user and open the configuration information of acquisition with flow point, export multiplexing data M uxDat then.At this,, can not use additional information adInf if do not need.In addition, can adopt the scheme such as MPEG-2 transmits stream (TS) or program flow (PS), or by using other predetermined scheme to sense data strOut2 packing, multiplexed to it then.For example, in Blu-ray disc (BD) standard, adopt the multiplexed sense data strOut2 of scheme that 4 byte stems is added to the MPEG-2TS packet that is called as source data packet (Source Packet), then storage.

Fig. 9 A shows from the structure example of the multiplex data of multiplex machine 800 outputs.

Shown in Fig. 9 A, management information and one or more encoding stream are stored in the multiplex data.In addition, handle each stream and make it can realize various player methods as one or more chips, for example summary is play and multi-angle is play.At this, chip illustrates the picture sequence among a picture or the random access units RAU, or the sequence of the random access units RAU of same stream.Fig. 9 B and 9C show the broadcast example.Particularly, Fig. 9 B shows the example that multi-angle is play.At stream 1 and stream N respectively under the situation with different angle store videos, can be by after the chip 1-1 of stream 1, playing the chip N-2 that flows N by changing angle, and after chip N-2 plays, return and flow 1 broadcast.Fig. 9 C shows the example that summary is play.For example, play typical scene by chip 1-1 and the chip 1-M that plays in the stream 1 selectively.

Figure 10 is that expression is carried out demultiplexing to multiplex data, with the operational flowchart of traditional Deplexing apparatus of obtaining coded data and broadcast encoder data.

At first, in step S201, Deplexing apparatus, obtains and the relevant information of one or more chips that will play to obtain management information the multiplex data demultiplexing, proceeds to step 204 then.The information relevant with chip comprises the time started of chip, the concluding time, is used at access information of chip Access Coding Minimal data etc.In

step

204 and 205, Deplexing apparatus decoding also shows picture in this chip, the final picture in this chip.At this, send under the situation that indication finishes the instruction of playing etc. in user operation, become in this instruction and finish to play when effective.

Figure 11 is the block diagram of the structure of the traditional Deplexing apparatus of expression.

Deplexing apparatus 900 comprises management information demultiplexing unit 21, chip information analysis unit 22, decoding unit 24 and demonstration 26.

Management information demultiplexing unit 21 is read multiplex data MuxDat from the multiplex data recording medium such as CD, the chip that analysis management information and determine according to user and so on or preordering method will be play.After this, management information demultiplexing unit 21 is to chip information analysis unit 22 output chip information Clip, as with the relevant information of determining of chip.

Chip information analysis unit 22 is used for the access information acs that access constitutes the picture of chip to decoding unit 24 outputs.On the other hand, decoding unit 24 is read video data Vdat according to access information acs from the multiplex data recording medium, outputs to display unit 26 to the data decode of reading with decoded result decOut.Display unit 26 is pressed DISPLAY ORDER and is shown decoded result.

MPEG-4AVC allows predict flexibly, and therefore the frame delay of chip is variable.Do not consider the frame delay of chip because traditional Deplexing apparatus changes chip, when change has the chip of different frame delay, in the demonstration at interval of picture, the gap occurs.

Figure 12 A to 12C shows the example that changes to the frame with two frame delay from the chip with a frame delay.

Figure 12 A shows the random access units RAU1-1 of the stream 1 with a frame delay, and Figure 12 B shows the random access units RAU2-1 of the stream 2 with two frame delay.At this, Figure 12 C shows decoding and the display timing generator when playing RAU2-1 after RAU1-1.

Because the frame delay of RAU1-1 is 1,, the top field P18 of RAU2-1 is decoded when in the decoding order that is presented at RAU1-1 during at last picture P15.Yet the frame delay of RAU2-1 is 2, and when being in second picture P2 decoding in to decoding order, the picture demonstration among the RAU2-1 does not still begin.Therefore, when being decoded, P2 do not have display frame.Therefore, in the demonstration between picture P15 and picture B0 at interval the gap appears.

Equally, after random access units RAU2-1, play under the situation of random access units RAU1-1, occur the gap in the code interval continuously showing to separate.In other words, in showing at interval crossover appears.Hereinafter, the gap in the DISPLAY ORDER is meant under the frame delay amount increase that connects and two kinds of situations that reduce and interruption occurs being connected.

As mentioned above, because traditional Deplexing apparatus can not show the picture that is positioned at the reformed part of chip with different frame delay, and the frame that is maintained fixed, therefore traditional multiplexed and Deplexing apparatus has the problem that the user that watches motion picture is felt under the weather.

Considered the present invention in order to address the above problem.The purpose of this invention is to provide a kind of that encoding stream and out of Memory is multiplexed to produce the multiplex machine of multiplex data, with the Deplexing apparatus that is used for the multiplex data demultiplexing is flowed with broadcast encoder, even the user is felt under the weather so that when any special play-back of carrying out such as multi-angle is play, also do not return.

Summary of the invention

To achieve these goals, multiplex machine of the present invention is multiplexing with one or more encoding streams and out of Memory, described encoding stream comprises can random-access access unit being the coded picture of unit, described multiplex machine comprises: coding unit, be used to produce encoding stream, so that in the process that two access units in all access units that comprise are decoded in order, the gap in the connection of two predetermined access unit, do not occur in to encoding stream; Multiplexed unit is used for carrying out the encoding stream of out of Memory and coding unit generation multiplexed.For example, in a first aspect of the present invention, coding unit produces encoding stream, is equal to each other the time lag when each retardation is the top field that plays when the top field the decoding order decode in the demonstration DISPLAY ORDER so that the retardation in two access units becomes.

Produce encoding stream so that will be carried out the retardation (frame delay) of two access units (for example, chip) of the special play-back mode that is equal to each other that becomes.Therefore, in the connection in these memory cell the gap does not appear in order to these access unit decodings the time.In other words, these access units seamlessly connect when carrying out special play-back each other.This means that frame frequency is constant.Therefore, can eliminate the uncomfortable feeling that causes to the user who watches the motion picture that constitutes by these access units.At this, for example, produce encoding stream, so that under situation, the retardation of these two access units is adjusted to predetermined delay amount according to the operation standard predetermined amount of delay of application program.

In addition, in a second aspect of the present invention, multiplex machine also comprises the deferred message generation unit, is used for producing and retardation delay associated information, in this device, multiplexed unit can carry out the deferred message of out of Memory and the generation of deferred message generation unit multiplexing.For example, in a third aspect of the present invention, the deferred message generation unit produces the deferred message of indication lag amount.As an alternative, in a third aspect of the present invention, the deferred message generation unit produces the mark that indication can seamlessly connect two access units, as deferred message.

Like this, under the situation of deferred message (frame delay information) indication lag amount, Deplexing apparatus can obtain retardation in these two access units at an easy rate by this deferred message.In addition, under the situation of indicating the retardation of these two access units to be equal to each other as the mark of deferred message, Deplexing apparatus can obtain the retardation in these two access units at an easy rate.Therefore, can make Deplexing apparatus carry out more accurate demultiplexing handles.

In addition, in a second aspect of the present invention, the mode that the gap can not appear with the connection of two access units in all access units that comprise in the encoding stream in coding unit produces encoding stream, and this connection is the variable-angle point.

Because these two access units have different angles, and are identical with previously described situation, in order to these access unit decodings the time, the gap does not appear in the connection of these access units.In other words, the mode with seamless multi-angle connects these access units.This means that frame frequency is immutable.Therefore, can eliminate the uncomfortable feeling that causes to the user who watches the motion picture that constitutes by these access units.

In addition, to achieve these goals, Deplexing apparatus multiplexed data of the present invention carries out demultiplexing, and described multiplex data comprises: (i) comprise can random-access access unit being the encoding stream of the coded picture of unit; (ii) with all access units in two predetermined access unit in retardation delay associated information, each retardation is the time lag when playing the top field that shows in the DISPLAY ORDER when the top field the decoding order is decoded, described Deplexing apparatus comprises: the deferred message demultiplexing unit, be used for the multiplex data demultiplexing, to obtain deferred message; And broadcast unit, be used for the deferred message that obtains during to the multiplex data demultiplexing according at the deferred message demultiplexing unit, decode in order and play access unit.

With (for example to these two access units, chip) retardation of the carrying out special play-back mode that is equal to each other that becomes produces under the example case of encoding stream, can decode and play these two access units in order, and not allow when special play-back, in the connection of these two access units, the gap not occur.In other words, can become and seamlessly connect these access units.This means that can to make frame frequency constant.The result is to eliminate the uncomfortable feeling that causes to the user who watches the motion picture that is made of these access units.In addition, under example case, can obtain the retardation of these two access units at an easy rate by deferred message by deferred message (frame delay information) indication lag amount.In addition, under the situation of usage flag indication lag information, the information that can obtain to indicate the retardation of these two access units to be equal to each other at an easy rate.Therefore, can carry out more accurate demultiplexing handles.

In addition, to achieve these goals, in multichannel multiplexing method of the present invention, to motion picture encoding, and the motion picture and the management information of coding is multiplexing, this multichannel multiplexing method comprises: produce one or more encoding streams; Obtain the frame delay in the encoding stream; Generation comprises the management of information information of the frame delay that is used to indicate acquisition; With carry out multiplexing to encoding stream and management information.In this multichannel multiplexing method, encoding stream is made of one or more random access units, the frame delay indication is respectively in these random access units, the retardation that occurs respectively during time period when playing the top field that shows in the DISPLAY ORDER when the top field the decoding order is decoded, and the frame delay in the encoding stream is variable.

In addition, management information can comprise the frame delay of encoding stream.

In addition, management information can comprise the maximum of frame delay in the encoding stream.

In addition, management information can comprise equal frame delay, and the common frame when being equal to each other as frame delay in the encoding stream postpones.

In addition, management information can comprise the frame delay of the broadcast unit that constitutes based on these random access units.

In addition, Deplexing method of the present invention is also to play the Deplexing method of the demultiplexed data that obtains to carry out demultiplexing according to the multiplexing data of multichannel multiplexing method.Described Deplexing method comprises: the frame delay of the encoding stream under these random access units that acquisition will be play; Frame delay when determining to play according to the frame delay that obtains; Play the picture that comprises in these random access units with the frame delay that basis is determined.

In addition, in determining the process of frame delay, the frame delay that can equate with the frame delay with the random access units that just has been played as current random access units is play these random access units of following after current random access units.

Should be noted that, the present invention not only can be as multiplex machine described above, Deplexing apparatus, multichannel multiplexing method and Deplexing method are realized, and to be used for the program of multiplex machine and Deplexing apparatus, store the recording medium of this program and realize by the multiplex data that multiplex machine produces.

So far, even, therefore can eliminate user's uncomfortable feeling owing to the present invention when special play-back also can guarantee to show with fixing frame frequency.Particularly, the present invention can improve the play quality of packet medium, and is therefore very practical.

The further information of relevant the application's technical background

Be incorporated herein the specification of submitting on June 2nd, 2004 that comprises, the whole content that accompanying drawing and claim disclose at the open No.2004-165030 of interior Japan Patent as a reference.

Description of drawings

From below in conjunction with the description of the drawings that shows particular embodiment of the present invention, can be well understood to these and other objects of the present invention, advantage and feature.In the accompanying drawings:

Figure 1A and 1B are the schematic diagrames that shows the MPEG-2 flow structure.

Fig. 2 A and 2B are the schematic diagrames that shows the gop structure among the MPEG-2;

Fig. 3 A and 3B are the schematic diagrames that shows the frame delay in the MPEG-2 stream;

Fig. 4 is the schematic diagram that shows the structure of MPEG-4AVC stream;

Fig. 5 is the schematic diagram that shows an example of predict in the MPEG-4AVC stream;

Fig. 6 A to 6C shows the schematic diagram that how to change the example of chip in the MPEG-4AVC stream;

Fig. 7 is the flow chart that shows the operation of traditional multiplex machine;

Fig. 8 is the block diagram that shows the structure of traditional multiplex machine;

Fig. 9 A to 9C is the schematic diagram of demonstration from the structure example of the data of traditional multiplex machine output;

Figure 10 is the flow chart that shows the operation of traditional multiplex machine;

Figure 11 is the block diagram that shows the structure of traditional multiplex machine;

Figure 12 A to 12C is the schematic diagram of the problem-instance that occurs when displaying the play multiplex data by the output of traditional multiplex machine;

Figure 13 is the flow chart that shows the operation of the multiplex machine in the first embodiment of the invention;

Figure 14 is the block diagram that shows the multiplex machine in the first embodiment of the invention;

Figure 15 A to 15E is the schematic diagram that shows the example of the data structure of being exported by the multiplex machine in the first embodiment of the invention;

Figure 16 is the flow chart that shows the operation of the multiplex machine in the second embodiment of the invention;

Figure 17 is the block diagram of the multiplex machine in the second embodiment of the invention;

Figure 18 is the flow chart that shows the operation of the Deplexing apparatus in the third embodiment of the invention;

The flow chart of the operation of Deplexing apparatus when Figure 19 is the frame delay that shows in determining in the third embodiment of the invention to play;

The flow chart of the operation embodiment of Deplexing apparatus when Figure 20 A to 20D is the frame delay that shows in determining in the third embodiment of the invention to play;

Figure 21 is the block diagram that shows the Deplexing apparatus in the third embodiment of the invention;

Figure 22 is the schematic diagram that shows the data level of HD-DVD;

Figure 23 is the schematic diagram that shows the logical space structure of HD-DVD;

Figure 24 is the schematic diagram that shows the structure of vob information file;

Figure 25 is the schematic diagram of time map;

Figure 26 is the schematic diagram of structure of listing file of displaying the play;

Figure 27 is the schematic diagram of demonstration corresponding to the structure of the program file of playlist;

Figure 28 is the schematic diagram that shows the structure of BD management information file;

Figure 29 is the schematic diagram that shows the structure of the program file be used to write down the clobal processor;

Figure 30 is the block diagram that shows the overview structure of HD-DVD player; With

Figure 31 A and 31B are the schematic diagrames of structure of the recording medium of the displayed record program that is used to realize multichannel multiplexing method of the present invention and Deplexing method; With Figure 31 C be declare record and the schematic diagram of playing the structure of the program on the dish flexibly.

Embodiment

Below with reference to the description of drawings embodiments of the present invention.

(first execution mode)

Figure 13 is the flow chart that shows the operation of the multiplex machine in the first embodiment of the invention.Multiplex machine output is through multiplexed data, can indicate the frame delay information of the encoding stream of storing in the multiplex data through multiplexing data.

At first, in step 301, multiplex machine produces the MPEG-4AVC encoding stream.In step 302, obtain frame delay, and proceed to step 303 at the encoding stream of step 301 generation.In step 303, judged whether to produce all encoding streams that will store in the multiplexed data, and the process of repeating step 301 and step 303, produced all encoding streams up to.In step 304, generation will be as the frame delay information of the management information of multiplex data storage, and proceed to step 305.In step 305, except the information that is produced by traditional multiplex machine, also generation is used to indicate frame delay management of information information.At last,, step 301 encoding stream that produces and the management information that produces in step 305 are carried out multiplexing, export multiplex data then in step 306.

Figure 14 is the block diagram that shows the structure of the multiplex machine in the first embodiment of the invention.

Multiplex machine 100A comprises coding unit 15, memory 12, and management information generation unit 16, frame delay obtains unit 17 and multiplexed unit 18.

The said modules that is noted that the multiplex machine 100A of present embodiment has been endowed identical reference number with same components in traditional multiplex machine shown in Figure 8, and no longer these assemblies is described.

The moving picture data Vin of 15 pairs of inputs of coding unit encodes, and the coded data strIn that obtains is stored in the memory 12.In addition, coding unit 15 obtains the frame delay frDly of unit 17 outputting encoded data strIn to frame delay.

Frame delay obtains unit 17 and produces the deferred message dlyInf that will be stored in the management information as frame delay information, then it is outputed to management information generation unit 16.

Management information generation unit 16 as sense data strOut1, outputs to it multiplexed unit 18 according to the coded data strIn that reads from memory 12 and the analysis result of deferred message dlyInf are produced management information bank then.

The 18 pairs of management information banks in multiplexed unit, the sense data strOut2 that reads from memory 12 and with coded data separate obtain, as the additional information adInf of information-setting by user, export multiplex data MuxDat then.Be noted that and the frame delay in when coding can be limited in predetermined value or be lower than predetermined value that this frame delay is provided with by coding unit 15.

Figure 15 A to 15E shows the structure embodiment by the multiplex data of multiplex machine 100A output.

Multiplex data shows the frame delay information that is stored in the chip in the multiplex data shown in Figure 15 A.At this, the storage number of chips is N in the multiplex data, and frame delay is from postponing 1 to postponing N.Being noted that each chip can be the unit of the zones of different in the indication same-code stream, perhaps can be the chip that belongs to different coding stream.

Figure 15 B to 15C shows the structure of frame delay information.Figure 15 B stores the frame delay of each chip as table information.Figure 15 C shows the maximum of the frame delay of chip.As maximum, the maximum in the frame delay of all chips of storing in the multiplex data can be shown, perhaps can provide maximum in the frame delay of the chip that will play in order according to playlist.In addition, predetermined value can be expressed as maximum.Figure 15 D shows the value of the common frame delay that uses in each chip.Under the frame delay of each chip is variable situation, can express these values.In addition, under the immutable situation of the frame delay of each chip, can express the frame delay that when playing, uses.Whether the frame delay that Figure 15 E shows chip is equal to each other.For example, the label information whether these frame delay are equal to each other is indicated in storage.The information shown in Figure 15 B to 15D that the is noted that use that to be combined.

Be noted that and express only relevant frame delay information with specific chip.At first, can provide and be used for the relevant frame delay information of chip of multi-angle broadcast or summary broadcast according to the player method of chip.In addition, can frame delay be set according to the attribute of top random access units in the chip.For example, be defined under the situation that the IDR picture changes angle the relevant frame delay information of chip that only can to provide with its top random access units be the IDR picture.In addition, can be only under the situation that the summary that guarantees seamlessly to connect each other chip is play, provide frame delay information.

In addition, can provide the frame delay information of the encoding stream of storing in the multiplex data, and directly not provide the frame delay information of each chip.At this moment, by the information of the encoding stream under each chip with the frame delay of indication encoding stream is associated, can provide the frame delay of each chip.Use this method under can two kinds of situations below: (i) under the immutable situation of frame delay of each chip in the same-code stream and (ii) provide under the peaked situation of frame delay of the chip that comprises in the same-code stream.

Under the situation of using the application standard such as Blu-ray disc (BD) or high definition (HD)-DVD, these standard definition frame delay should be equal to each other, or maximum or predetermined value between the definition frame delay, because this application standard has provided the information relevant with frame delay, be noted that and storage frame postpone as management information.

In addition, utilizing the agreement such as TS or real-time transport protocol (rtp) to receive under the situation of multiplex data, can obtain frame delay information as Play Control information through network.For example, to playback terminal notice Play Control information the time, use imprinting paragraph (session) to describe under the situation of agreement (SDP), can describe the frame delay information among the SDP.In addition, the frame delay information by in the scene description language of indication such as synchronized multimedia integration lang (SMIL) can postpone to the playback terminal notification frame.

In addition, can connect indication frame delay information on the random-access basis an arbitrary access.In addition, can come indication frame delay information in encoding stream by the top field that for example it is added among the random access units RAU.

Like this, by indicating frame delay information by the management information in the multiplex data of multiplex machine output.Therefore, when playing multiplex data, regulate frame delay it can be flowed by broadcast encoder, and do not allow when changing chip, in demonstration, to occur the gap.

(second execution mode)

Figure 16 is the flow chart that shows the operation of the multiplex machine in the second embodiment of the invention.

At first, in step 401, the multiplex machine setting is stored in the value of the common frame delay that uses of encoding stream in the multiplex data.In step 402, produce the MPEG-4AVC encoding stream according to the frame delay that is provided with in step 401.In step 403, judged whether to produce all encoding streams that will store in the multiplexed data, and the process of repeating step 402, up to all encoding streams are encoded.In step 404, generation will be as the frame delay information of the management information of multiplex data storage, and proceed to step 405.In step 405, except the information that is produced by traditional multiplex machine, also generation is used to indicate frame delay management of information information.At last, in step 406, step 401 encoding stream that produces and the management information that produces in step 405 are carried out multiplexing, the multiplex data that obtains of output then

Figure 17 is the block diagram that shows the structure of the multiplex machine in the second embodiment of the invention.

Multiplex machine 100B comprises frame delay determining unit 41, coding unit 42, memory 12, management information generation unit 16 and multiplexed unit 18.

The said modules that is noted that the multiplex machine 100B of present embodiment has been endowed identical reference number with same components in traditional multiplex machine shown in Figure 8, and no longer these assemblies is described.

Frame delay determining unit 41 is determined the frame delay of encoding stream, and to coding unit 42 and management information generation unit 16 output delay information dlyInf.At this frame delay of determining can be the predetermined value of application standard, perhaps can be at the multiplex machine setting, or be provided with by the user.Coding unit 42 comes the moving picture data Vin of input is encoded according to the frame delay of deferred message dlyInf indication, then coded data strIn is stored in the memory 12.In other words, coding unit 42 is so that the mode that the frame delay in the unit of random access units is equal to each other produces one or more coded data strIn, so that in order to the specific access units decoding of two of the access unit (its coded data or chip) that comprises in as one or more coded data strIn the time, make in the connection of these access units the gap not occur.Be noted that preventing to occur in the connection of these access units of when decoding the gap is meant in the demonstration at interval that prevents picture and the gap occurs, prevents that crossover from appearring in separating of picture in the code interval.

Management information generation unit 16 is according to the coded data strOut1 that reads from memory 12 and the analysis result of deferred message dlyInf are produced management information bank, as sense data strOut1 and deferred message dlyInf, then management information bank is outputed to multiplexed unit 18.

Multiplexed unit 18 multiplexing (i) management holder, the (ii) sense data strOut2 that has read from memory 12, (iii), export the multiplex data MuxDat that obtains then such as the additional information adInf that by information-setting by user, separately obtains from coded data.

Be noted that under the situation by pre-defined frame delay such as application standards multiplex machine 100B can have the structure that does not comprise frame delay determining unit 41, what can replace is that coding unit 42 is carried out encoding process according to fixing frame delay.In encoding stream under the constant condition, can in management information, not comprise the frame delay when determining to play under the situation of frame delay information at frame delay.Therefore, need always not indicate frame delay information in management information.

Owing to the frame delay in the encoding stream that will be stored in the multiplex data under the situation of being exported multiplex data by multiplex machine by this way is immutable, so the frame delay of each chip also is immutable.Therefore, even changing under the situation of chip, also can be when beginning to play in demonstration broadcast encoder stream seamlessly, and do not need to regulate frame delay.

Be noted that in the above-described embodiment, can adopt the encoding scheme except that MPEG-4AVC, as long as its definition frame delay is variable.

In addition, the frame delay of these random access units (encoding stream or chip) that can play in order in seamless link or seamless multi-angle broadcast is immutable.At this, seamless link is meant and seamlessly connects the chip that comprises in same stream or the various flows.In addition, seamless multi-angle is play and is meant chip that comprise, that its angle differs from one another in the connection encoding stream, and changes these angles.For example, can between from the outset, during the 30th second time period, adopt a kind of angle, and during the 30th second to the 60th second time period, select a plurality of angles to play.At this moment, each chip of expression different angles has identical frame delay.In addition, can make the frame delay of each angle constant, play irrelevant and whether can seamlessly carry out multi-angle with it.This be because, frame delay is that the reproduction quality is changeable under the unit situation about changing with the angle when multi-angle is play.Be noted that and carry out this seamless link or seamless multi-angle broadcast to encoding stream.

Be noted that each access unit that can require to use in seamless link or the multi-angle broadcast satisfies following condition: can be with reference to not coming any picture codec in first access unit than the picture in early second access unit of decoding of first access unit.Access unit like this is corresponding to the sealing GOP among the MPEG-2, or the random access units RAU that begins with the IDR picture among the MPEG-4AVC.Therefore, can allow to reference to the picture in the same access unit immutable to its frame delay of forming in the access unit of picture codec.

In addition, under the immutable situation of frame delay of these access units of playing in order in making above-described seamless link etc., multiplex machine is prejudged and whether should be carried out seamless link or seamless multi-angle to these access units that should produce and play.After this, judge that at multiplex machine these access units should be under the situation of target, so that the mode that the frame delay of target random access units is equal to each other encodes, to produce encoding stream.For example, under the situation of two encoding streams being carried out seamless link etc., produce encoding stream backward, so that the frame delay of encoding stream equates with the frame delay of encoding stream forward backward.

In addition, the frame delay information that comprises in the management information bank can pointer is common to the frame delay of the specific access unit such as seamless link (encoding stream or chip).In addition, also can be the target that the indication access unit can become seamless link, the mark of multi-angle broadcast etc.Such mark is stored in playlist, in the time map etc., as the information of indication connection attribute.

(the 3rd execution mode)

Figure 18 is the flow chart of the operation of the Deplexing apparatus in the expression third embodiment of the invention.The multiplexed data that Deplexing apparatus input and broadcast are produced by the multiplex machine in first and second execution modes.

At first, in step 501, Deplexing apparatus with acquisition management information, and obtains the information relevant with the chip that will play to the multiplex data demultiplexing.According to by the instruction from the user, or predetermined playing sequence such as the playlist in the management information is determined the chip that will play.

Next, in step 502, obtain to be confirmed as to play the chip frame delay information of chip, and proceed to step 503 in step 501.In step 503, according to the frame delay of determining in the frame delay information of step 502 acquisition when playing chip.

In step 504 and step 505, decoding also shows picture in the chip, last picture in chip.At this, under situation about play finishing, become in this instruction and to finish to play when effective by user's operational order.Be noted that in step 501 to step 503, only when change belongs to the chip of different coding stream, can determine frame delay.

Figure 19 is the flow chart of the operation when determining frame delay during the broadcast that shows in the step 503 of Figure 18.

At first, in step 601, Deplexing apparatus judges whether the frame delay that is included in the encoding stream in the demultiplexed data is immutable.Under the immutable situation of frame delay, flow process proceeds to step 603.On the contrary, under the variable situation of frame delay, flow process proceeds to step 602.

In step 603, determine to play the encoding stream in the multiplex data with common frame delay.In step 602, judge whether the frame delay of the chip that will play in order is immutable.Under the immutable situation of frame delay, flow process proceeds to step 604.On the contrary, under the variable situation of frame delay, flow process proceeds to step 605.In step 604, determine to come broadcast encoder stream according to the frame delay of top chip in playing.In step 605, determine to regulate the frame delay in playing.The method of regulating frame delay in the broadcast that is used in is described below.

1. the largest frames of regulating in the frame delay that will play chip postpones.

2. the frame delay of the chip of just having play.

3. use predetermined frame delay.

Under the situation of the chip that can determine to play, first method is effective especially.Under the situation of the chip that the dynamic change by user instruction will be play, second method is effective especially.In addition, can be from the management information of multiplex data, or the information in the encoding stream, or obtain by application standard under the peaked situation of frame delay, the third method is effective.In addition, can use is the frame delay that this device is scheduled to.

Figure 20 C and 20D show the embodiment of first and second kinds of methods respectively.Figure 20 C shows the embodiment that plays the chip 2 with two frame delay after the chip 1 with a frame delay.At this, the frame delay when playing chip 1 is 2.Figure 20 D shows the embodiment that is changed to the chip 1 with a frame delay when playing the chip 2 with two frame delay by user instruction.At this moment, the frame delay of chip 1 is 1 at first, according to the frame delay of chip 2, is under 2 the prerequisite it to be play at the frame delay of chip 1.Determine frame delay by this way, make it possible to the gap not occur in the demonstration interval of the position that when broadcast encoder flow, between chip 1 and chip 2, changes.

Be noted that when the frame delay that may have the encoding stream in the multiplex data is equal to each other, do not provide the situation of frame delay information in the demultiplexed data, as shown in second execution mode.Under the situation of such broadcast multiplex data, do not need the process in the step 502.In addition, preferably, determine when in step 503, beginning to show under the situation of frame delay, regulate the frame delay of top chip in playing.

In addition, under the peaked situation that is postponed by definition frames such as application standards, Deplexing apparatus can be play multiplex data according to predefined maximum always.

Figure 21 is the block diagram of structure that shows the Deplexing apparatus of the 3rd execution mode.

Deplexing apparatus 200 comprises management information demultiplexing unit 51, chip information analysis unit 52, frame delay determining unit 53, decoding unit 54 and display unit 55.

Management information demultiplexing unit 51 is read multiplex data MuxDat from the multiplex data recording medium such as CD, analysis management information, and determine the chip that will play according to user instruction or preordering method.After this, management information demultiplexing unit 51 is to chip information analysis unit 52 output chip information Clip, as with the relevant information of determining of chip.

Chip information analysis unit 52 is used for the access information acs that the picture of chip is formed in access to decoding unit 24 outputs.In addition, chip information analysis unit 52 obtains the deferred message dly of the chip that will play, and deferred message dly is outputed to frame delay determining unit 53.

Decoding unit 24 is read video data Vdat according to access information acs from the multiplex data recording medium, to video data Vdat decoding, then to display unit 54 output decoders decOut as a result.

Frame delay when frame delay determining unit 53 is determined to play, and to display unit 54 output delay T.

Display unit 54 comes display frame according to postponing T.Be noted that to exist shown in second execution mode, under the situation that the frame delay of the encoding stream in multiplex data is equal to each other, do not provide the situation of frame delay information in the multiplex data.Under the situation of such broadcast multiplex data, Deplexing apparatus can have the structure that does not comprise frame delay determining unit 53.

In addition, these access units are being carried out seamless link or multi-angle when playing, though the frame delay of the continuous broadcast unit that provides in playlist etc. is immutable, under the situation that does not guarantee seamless link, the frame delay in the junction is variable.Management information at multiplex data does not comprise under the situation of the information of indicating frame delay at this moment, can obtain the frame delay of access unit in decoding unit 24, can be input to frame delay determining unit 53 to the frame delay that obtains.At this, in the immutable broadcast area of frame delay, determine this frame delay according to the frame delay of the access unit at top in the playing sequence.On the contrary, in the variable broadcast area of frame delay, the frame delay determining unit is regulated the frame delay when playing.Be noted that in above-mentioned any situation, can use with application standard etc. in the identical frame delay of maximum of the frame delay determined.

(the 4th execution mode)

Function such as multi-angle is play the summary broadcast is to being used to play the optical disc apparatus particular importance of package media.Illustrate that multiplex data with the multiplex machine in above-mentioned first and second execution modes output records the example in the Blu-ray disc (BD) as CD of future generation here.

The record format of BD-ROM at first, is described.

Figure 22 is the structure of indication BD-ROM, particularly as the structure of the BD dish 104 of dish medium be stored in the schematic diagram of the data in the dish.The data that are stored in the BD dish 104 comprise AV data 103, the BD management information 102 such as the management information of relevant AV data and AV playing sequence and realize interactive BD playing program 101., for convenience, the explanation of BD dish is concentrated on the AV application of the Voice ﹠ Video content that is used for movie here, but can be to the same explanation of another kind application carrying out.

Figure 23 is the structural representation that shows the catalogue file that is stored in the logical data in the above-mentioned BD dish.For example, as DVD, CD etc., BD dish has the recording areas of thoughtful its periphery in it, and have interior week read in and the reading of periphery between, the logical address space that is used for the stored logic data.In addition, reading in the inboard, having and be called pulse cut section (Burst Cutting Area) (BCA), the exclusive district that can only read by driver.Because this district can not read from application, thus it can be used for, for example, copyright protection technology.

Filesystem information (volume) is stored in logical address space foremost, and the application data of video data and so on also is stored in here.As described in the background art, file system is for example UDF or ISO9660, and its allows to read the logical data that utilizes bibliographic structure or file structure storage as the situation of normal PC.

In this embodiment, as bibliographic structure on the BD dish and file structure, and then the BDVIDEO catalogue places under the path catalogue (ROOT (root)).This catalogue is the catalogue of the data of storage such as management information of perhaps handling in BD in the AV (illustrated in fig. 22 101,102 and 103) and so on.

Under the BDVIDEO catalogue, writing down following seven files.

(i) BD.INFO (filename is fixed), this is one " a BD management information ", and is the file of the information of the relevant whole BD dish of storage.The BD player is at first read this file.

(ii) BD.PROG (filename is fixed), this is one of " BD playing program ", and is the file of the Play Control information of the relevant whole BD dish of storage.

(iii) XXX.PL (" XXX " is variable, and extension name " PL " is fixed), this is one " a BD management information ", and is the file of storage as the playlist information of script (play sequence).Each playlist has a file.

(iv) XXX.PROG (" XXX " is variable, and extension name " PROG " is fixed), this is one of " BD playing program ", and the file of the Play Control information that to be storage prepare according to playlist.Corresponding playlist is to discern according to file body name (according to the coupling of " XXX ").

(v) YYY.VOB (" YYY " is variable, and extension name " VOB " is fixed), this is one of " AV data ", and is the file of storage VOB (identical with the VOB that illustrates in the background technology).Each VOB has a file.

(vi) YYY.VOBI (" YYY " is variable, and extension name " VOBI " is fixed), this is one " a BD management information ", and is the file of the flow management information of the relevant VOB as the AV data of storage.Corresponding playlist is to discern according to file body name (according to the coupling of " YYY ").

(vii) (" ZZZ " is variable to ZZZ.PNG, and extension name " PNG " is fixed), this is one of " AV " data, and is the file of storing the view data PNG (this is by the standardized picture format of V3C, reads and makes " ping ") that is used to constitute subtitle and menu.Each PNG image has a file.

The structure (BD management information) of BD navigation data is described below with reference to Figure 24 to Figure 29.

Figure 24 is the schematic diagram that shows the internal structure of VOB management information file (" YYY.VOBI ").

VOB management information has stream attribute information (Attribute) and the time map table (TMAP) of VOB.Stream attribute has video attribute (Video) and audio attribute (Audio#0 to Audio#m).More particularly, under the situation of audio stream, because VOB has a plurality of audio streams simultaneously, so the number (Number) by audio stream comes the existence of indicator data field or do not exist.

Be to be stored in the value that video attribute (Video) in the field and corresponding field may have respectively below.

(i) compressed format (Coding): MPEG-1; MPEG-2; MPEG-4; And MPEG-4AVC (advanced video coding).

(ii) resolution (Reslution): 1920 x 1080; 1440 x 1080; 1280 x 720; 720 x 480; With 720 x 565.

(iii) the ratio of width to height (Aspect): 4 to 3; With 16 to 9.

(iv) frame rate (Framerate): 60; 59.94 (60/1.001); 50; 30; 29.97 (30/1.001); 25; 24; With 23.976 (24/1.001).

Below be to be stored in the value that audio attribute (Audio) in the field and respective field may have respectively.

(i) compressed format (Coding): AC3; MPEG-1; MPEG-2; And LPCM.

(ii) sound Taoist monastic name (Ch): 1 to 8.

(iii) linguistic property (Language):

Time map table (TMAP) is the table that is used to store based on the information of VOBU, and has VOBU quantity and corresponding VOBU information bar (VOBU#1 to VOBU#n) that VOB has.Each VOBU information bar comprises as the I_start of the address (start address of I picture) of the packet of TS foremost of VOBU with up to the play start time (PTS) that departs from address (I_end) and I picture of the end address of I picture.

Figure 25 is the schematic diagram that shows the VOBU details.

As everyone knows, owing to can carry out variable bit-rate compression so that recorded video stream in high quality to mpeg video stream, so do not have direct ratio between reproduction time and the size of data.On the other hand, owing in as the AC3 of audio compress standard, carry out the fixed bit rate compression, so can obtain relation between time and the address from primary expression.But, under the situation of MPEG video data, each frame has the fixing demonstration time, for example, and under the NTSC situation, frame has 1/29.97 second demonstration time, but according to characteristics of image, or the picture type that uses in the compression, for example, I picture, P picture or B picture compress each frame size of data afterwards and greatly change.Therefore, under the situation of mpeg video stream, can not utilize the relation between primary expression express time and the address.

As expected, in the mpeg system stream of multiplexed MPEG video data, that is, in VOB, can not utilize the relation between primary expression express time and the size of data.Therefore, in VOB, time and address are linked together with time mapping table (TMAP).

By this way, under the situation that has provided temporal information, the VOBU (following the PTS of VOBU in order) under search time at first, make the PTS that is close to before this time jump into the VOBU (by the address of I_start appointment) that TMAP has, begin decoding from the picture of I foremost of this VOBU, and begin to show from picture corresponding to this time.

Next, the internal structure of playlist information (" XXX.PL ") is described with reference to Figure 26.

Playlist information comprises unit list (CellList) and list of thing (EventList).

Unit list (CellList) is the broadcast unit sequence in the playlist, and each unit is with the regulation played in order of indication in this tabulation.The content of unit list (CellList) is the information (Cell#1 to Cell#n) of unit number (Number) and each unit.

Unit information (Cell#) has vob file name (VOBName), time started among this VOB (In) and concluding time (Out), and subtitle (SubtitleTable).In each VOB, time started (In) and concluding time (Out) are expressed as frame number.Utilize above-mentioned time map table (TMAP) can obtain to play the address of required VOB data.

Subtitle table (SubtitleTable) is the table of the subtitle information of storage and VOB synchronous playing.As the situation of audio frequency, multilingual is included in the subtitle.The first information of subtitle table (SubtitleTable) comprises language number (Number) and the table of preparing according to language subsequently (Language#1 to Language#k).

Each language table (Language#) comprises language message (Lang), independent displaying subtitle the subtitle information bar number (Number) and want the subtitle information (Speech#1 to Speech#j) of the subtitle of independent displaying.Subtitle information (Speech#) comprises image data file name (Name), and subtitle shows the time started (In), and subtitle shows concluding time (Out) and subtitle display position (Position).

List of thing (EventList) is the table that defines each incident that takes place in the playlist.List of thing comprises event number (Number) and each incident (Event#1 to Event#m).Each incident (Event#) comprises event type (Type), event id (ID), Time To Event (Time), and incident duration (Duration).

Figure 27 is the button.onrelease table (" XXX.PROG ") with button.onrelease (this is to be used for time-event and the customer incident that menu is selected) of preparing according to playlist.

The button.onrelease table comprises the button.onrelease/program number (Number) and the events corresponding handling procedure/program (Program#1 to Program#n) of definition.The content of each button.onrelease/program (Program#) is the definition (＜event_handler〉tag) of the beginning of button.onrelease and forms right button.onrelease ID (ID) with the event id of previous explanation that and then it is following the program that illustrates in function (Function) back " { } ".Be stored in incident (Event#1 to Event#m) in the list of thing (EventList) of " XXX.PL " of previous explanation is to utilize ID (ID) appointment of the button.onrelease of " XXX.PROG ".

Next, the internal structure of the information (" BD.INFO ") of relevant whole BD dish is described with reference to Figure 28.

The event table (EventList) that comprises header list (TitleList) and clobal about the information of whole BD dish.

Header list (TitleList) comprises the title number (Number) of dish and follows many heading messages (Title#1 to Title#n) in the title number back.Each bar heading message (Title#) comprises the play sequence table (PLTable) that is included in the title and the Zhang Liebiao (ChapterList) in the title.Play sequence table (PLTable) comprises number (Number) of the playlist in the title and as the names of playlists (Name) of the filename of playlist.

Zhang Liebiao (ChapterList) comprises chapter number (Number) and many chapter information (Chapter#1 to Chapter#n) that are included in the title.Every chapter information (Chapter#) comprises the cell list (CellTable) that is included in this chapter, and cell list (CellTable) comprises unit number (Number) and many unit item information (CellEntry#1 to CellEntry#k).Unit item information (CellEntry#) comprises that names of playlists, names of playlists comprise unit and the unit number in this playlist.

List of thing (EvenList) comprises clobal number (Number) and many clobals.Should be noted that at first the clobal of definition is called first incident (FirstEvent), and be that BD inserts the incident of at first calling behind the player.The event information of clobal only has an event type (Type) and an event id (ID).

Figure 29 is the table (" BD.PROG ") of the program of clobal handling procedure.The content of the button.onrelease table that illustrates among the content of this table and Figure 27 is identical.

The multiplexing dateout of the multiplex machine in first and second execution modes is used under the situation that resembles above-mentioned such BD-ROM, and VOBU is made up of one or more random access units RAU, has stipulated the playing sequence of chip in playlist.At this, can provide the frame display message by BD management information.For example, can be in the playitem of playlist the frame delay information stores, or in the table of the address information indication such as the EP mapping.Frame delay information also can be stored in the table of the attribute information of indicating encoding stream.In addition, be stored in the maximum of the frame delay of the encoding stream in the multiplex data, (ii) between all encoding streams etc., under the situation of common frame delay, can indicating frame delay as the information higher than the information relevant with each encoding stream in indication (i).

Be noted that the playing sequence that to determine chip according to the information different with playlist or predefined procedure.

Be noted that the access information such as EP mapping to be stored in the table, or be the form of the text data that can be XML (extend markup language) etc. as binary data.

(the 5th execution mode)

Figure 30 is the block diagram that the functional structure of the player of playing the BD dish relevant with the 5th execution mode roughly is shown.

Read data on the BD dish 201 by optical picking-up head 202.According to the type of corresponding data, the data of reading are sent to private memory.BD playing program (content of " BD.PROG " or " XXX.PROG ") is sent to program storage 203.In addition, BD management information (" BD.INFO ", " XXX.PL " or " YYY.VOBI ") is sent to management information memory 204.Also AV data (" YYY.VOB " or " ZZZ.PNG ") are sent to AV memory 205.

The BD playing program that is recorded in the program storage 203 is handled in routine processes unit 206.In addition, management information is handled unit 207 and is handled the BD management information that is recorded in the management information memory 204.Manifest processing unit 208 and handle the AV data that are recorded in the AV memory 205.

The information of the playlist that routine processes unit 206 receiving management information process units 207 will be play and the event information such as program implementation regularly, and the processing of executive program.In addition, the playlist that will play of reprogramming dynamically.This can send to management information processing unit 207 by the play instruction with playlist and realize.Routine processes unit 206 receives the incident from the user, that is to say, receives request by remote controller, and under situation about having corresponding to the program of customer incident, carries out this program.

Management information is handled the instruction that unit 207 receives from routine processes unit 206, analyzes playlist and corresponding to the management information of the VOB of playlist, and instruction manifests processing unit 208 and plays target AV data.Management information is handled unit 207 and is also received from the standard time information that manifests processing unit 208, and instruction manifests processing unit 208 and stops to play the AV data according to temporal information.In addition, management information is handled unit 207 and is produced an incident, regularly is notified to routine processes unit 206 so that program is carried out.

Manifest processing unit 208 and have the decoder that to handle video, audio frequency, subtitle/image (still frame) respectively.It is decoded and output AV data according to the instruction of handling unit 207 from management information.At video data, and under the situation of subtitle/image,, be provided to then on the corresponding private use plane they decodings, that is, and on the video plane 210 and the plane of delineation 209.After this, 211 pairs of videos of synthetic processing unit are carried out the synthetic display unit of handling and video being outputed to TV and so on.

Play or summary when playing in multi-angle, manifest the operation that multi-angle is play or summary is play of processing unit 208 translation users requests, and the information relevant with variable-angle point etc. is notified to management information handles unit 207.Frame delay when management information is handled unit 207 and determined broadcast according to the frame delay information of the chip that will play, and this frame delay is notified to manifests processing unit 208.

Access information such as EP mapping can be stored in the table as binary data, or be the form of the text data that can be XML (extend markup language) etc.

(the 6th execution mode)

In addition, by the program that is used for realizing multiplexing method that above-mentioned execution mode provides and Deplexing method being recorded the recording medium such as floppy disk, can independently carry out the processing that provides in each execution mode in the computer system at an easy rate.

Figure 31 C shows computer system and how to use the program that writes down in the recording medium such as floppy disk to carry out motion picture encoding method and motion picture coding/decoding method in the present embodiment.

Figure 31 A shows the example as the physical format of the floppy disk of recording medium.Figure 31 B shows the positive schematic diagram and the cross-sectional view of floppy disk and this floppy disk outward appearance.Floppy disk (FD) is contained among the shell F, be formed with a plurality of tracks (Tr) with one heart on the surface of Zhou Zaipan in dish outer thoughtful, and each track is divided into 16 sectors (Se) on angle direction.Therefore, under the situation of floppy disk storage said procedure, program is stored on the floppy disk (FD) and distributes in its district.

In addition, Figure 31 C shows the structure of the program on record and the broadcast floppy disk.Under situation about the said procedure that is used to realize multichannel multiplexing method and Deplexing method being recorded on the floppy disk FD, computer system Cs writes on program on the floppy disk by floppy disk.Program structure in utilizing floppy disk is above-mentioned to be used to realize under the situation of the multiplex machine of multichannel multiplexing method and Deplexing method and Deplexing apparatus, from the floppy disk read routine, and it is sent to computer system by floppy disk.

Should be noted that above-mentioned explanation utilizes floppy disk to carry out as recording medium, still, program can be recorded on the CD.In addition, recording medium is not limited thereto, and also can use such as IC-card, and the other recording medium of ROM box and so on is as long as it can logging program.

So far, the multiplex machine relevant with the present invention has been described according to above-mentioned various execution modes, Deplexing apparatus, BD disk player etc., but the invention is not restricted to these execution modes.The present invention includes the various changes that those skilled in the art can expect according to these execution modes, and these change in the scope of technical theme of the present invention.

For example, the present invention includes with lower device: (i) optical disc recording apparatus, this optical disc recording apparatus comprises the multiplex machine in the present embodiment, the motion picture dispensing device; The digital television broadcasting device; Web server; Communicator, personal digital assistant device etc.; (ii) motion picture receiving system, this motion picture receiving system comprises the Deplexing apparatus in the present embodiment; The digital television broadcasting receiving system; Communicator; Personal digital assistant device, or the like.

Each functional block that should be noted that the block diagram of (Figure 14, Figure 17 and Figure 21 etc.) generally realizes with the LSI as the large scale integrated circuit of integrated circuit.Each functional block can be made for chip.In addition, can be with one, some, or the repertoire piece is integrated in the chip and (for example, the functional block except memory can be manufactured a chip).

, circuit is called LSI here, still,, can be referred to as integrated circuit (IC), system LSI, super LSI or super large LSI according to integrated level.

In addition, the method that manufactures integrated circuit is not limited thereto, and also can realize by special circuit or general processor.In addition, after making the reconfigurable processor that LSI maybe can reconfigure the connection of the circuit unit among the LSI or setting, but can use programmable field programmable gate array (FPGA).

In addition, further develop or any derivation technology occurred at semiconductor technology, when causing can considering to substitute the method for manufacturing integrated circuit of LSI, nature can be with integrated these functional blocks of this technology.The application of biotechnology is possible.

In addition, in each functional block, the device of the configuration store data that will encode or write down independently, rather than be integrated in the chip.

Although below only describe example embodiment of the present invention in detail, those skilled in the art should be known in and can carry out multiple modification and not break away from new instruction of the present invention and advantage example embodiment.Therefore, all such modifications comprise within the scope of the invention.

Industrial applicibility

Multichannel multiplexing method of the present invention and Deplexing method can be applied to have such as multi-angle broadcast Put and the general device of the trick play functionality play and so on of making a summary, so they are at multichannel MPEG-4 Especially effective in the broadcast of the package media of AVC encoding stream.

Claims

1. multiplex machine is used for one or more encoding streams and another information multiplexedly, and described one or more encoding streams comprise can random-access access unit, and each access unit is made of coded picture, and described multiplex machine comprises:

Coding unit, be used for the time successively to two access unit decodings, by the retardation in described each access unit is equated, make the mode that the gap between the picture displayed of the junction of two access units, do not occur produce described one or more encoding stream, described two access units are to select in the access unit that comprises from described one or more encoding streams, the time delay when each indication of described retardation is played when by decoding order top field decode by DISPLAY ORDER demonstration top field; With

Multiplexed unit, be used for described another information and described one or more encoding stream multiplexed, described another information comprises whether described two access units of indication can be by the marks of seamless link, and described one or more encoding streams are produced by described coding unit.

2. multichannel multiplexing method is used for one or more encoding streams and another information multiplexedly, and described one or more encoding streams comprise can random-access access unit, and each access unit is made of coded picture, and described multichannel multiplexing method comprises:

Coding step, be used for the time successively to two access unit decodings, by the retardation in described each access unit is equated, make the mode that the gap between the picture displayed of the junction of two access units, do not occur produce described encoding stream, described two access units are to select in the access unit that comprises from described one or more encoding streams, the time delay when each indication of described retardation is played when by decoding order top field decode by DISPLAY ORDER demonstration top field; With

Multiplexed step, be used for described another information and described one or more encoding stream multiplexed, described another information comprises whether described two access units of indication can be by the marks of seamless link, and described one or more encoding streams produce in described coding step.

3. a Deplexing apparatus carries out demultiplexing to multiplexed data, and described Deplexing apparatus comprises:

The mark demultiplexing unit is used for going out mark from the multiplexed data demultiplexing, and described multiplexed data comprises:

Have can random-access access unit one or more encoding streams, each access unit is made of coded picture; With

Another information, described another information comprise that indication is continuously to two in the access unit

During the decoding of individual access unit, mark that whether can described two access units of seamless link;

Determining unit, be used for when can seamless link by the indication of the mark of described mark demultiplexing unit demultiplexing described two access units, determine to play described two access units the time delay between the demonstration time of the picture that at first is shown in the decode time of at first decoded picture and the access unit that at first is played in the access unit that described decoding delay indication at first is played according to decoding delay; With

Broadcast unit is used to utilize described decoding delay, decodes successively and shows described two access units.

4. a Deplexing method carries out demultiplexing to multiplexed data, and described Deplexing method comprises:

Mark demultiplexing step is used for going out mark from the multiplexed data demultiplexing, and described multiplexed data comprises:

Another information, described another information comprise indication continuously to two access units decodings in the access unit time, mark that whether can described two access units of seamless link;

Determining step, in described mark demultiplexing step during can seamless link described two access units of the mark of demultiplexing indication, determine to play described two access units the time delay between the demonstration time of the picture that at first is shown in the decode time of at first decoded picture and the access unit that at first is played in the access unit that described decoding delay indication at first is played according to decoding delay; With

Play step, utilize described decoding delay, decode successively and show described two access units.

One kind in computer readable recording medium storing program for performing record comprise the method for the multiplexed data of one or more encoding streams and another information, described one or more encoding stream comprises can random-access access unit, each access unit is made of coded picture, and described method comprises:

Coding step, be used for the time successively to two access unit decodings, by the retardation in each access unit is equated, make the mode that the gap between the picture displayed of the junction of two access units, do not occur produce described encoding stream, described two access units are to select in the access unit that comprises from described one or more encoding streams, the time delay when each indication of described retardation is played when by decoding order top field decode by DISPLAY ORDER demonstration top field; With

Multiplexed step, be used for described another information and described one or more encoding stream multiplexed, described another information comprises whether described two access units of indication can be by the marks of seamless link, and described one or more encoding streams produce in described coding step;

Recording step is recorded in multiplexing multiplexed data in the described multiplexed step in described recording medium.

6. a demultiplexing system is used for reading multiplexed data and the multiplexed data demultiplexing to reading from the computer readable recording medium storing program for performing that records multiplexed data,

The multiplexed data that writes down in the wherein said recording medium comprises:

One or more encoding streams, described one or more encoding streams have can random-access access unit, and each access unit is made of coded picture; With

Another information, described another information comprise the mark that indication whether can described two access units of seamless link successively to the decoding of two access units in the access unit time,

Read multiplexed data from described recording medium, and the demultiplexing system of the multiplexed data demultiplexing that read comprised:

The mark demultiplexing unit is used for the multiplexed data demultiplexing that reads is gone out mark;