CN1941905A

CN1941905A - Encoding/decoding apparatus and encoding-multiplexing/decoding-demultiplexing apparatus

Info

Publication number: CN1941905A
Application number: CNA2006100957973A
Authority: CN
Inventors: 渡边敏明; 菊池义浩; 中条健; 永井刚
Original assignee: Toshiba Corp
Current assignee: Toshiba Corp
Priority date: 1996-09-02
Filing date: 1997-09-02
Publication date: 2007-04-04
Also published as: CN1941904A; CN1925611A; CN101094399A; CN101094210A

Abstract

In an information transmission method, error robustness is provided for the bit stream itself so that decoding processing can be properly performed even in the event of an error in important information such as header information. A bit stream reconstruction circuit ( 107 ) in an encoding apparatus adds sync signals to the heads of encoded data streams, which are encoded by an encoder ( 103 ), in certain bit stream units, and then inserts designation information in each bit stream by using a designation information insertion circuit ( 106 ). Each designation information indicates the addition of information for reconstructing important header information. By inserting the designation information in the bit stream obtained, reconstruction information can be added to the bit stream. Even if, therefore, an error is introduced into the header information, and the information cannot be used for decoding processing, the decoding processing can be properly continued by using the new reconstruction information designated by the designation information as a substitute.

Description

Coding/decoding apparatus and encoding-multiplexing equipment and decoding/demultiplexing device

The application is to be on September 2nd, 1997 applying date, and name is called the dividing an application of No. 200410034237.8 patent applications of " information transferring method, coding/decoding apparatus and code multiplexing/decoding demultiplexing equipment ".

Technical field

The present invention relates to wireless communication networks such as ISDN, the internet or as the radio communication net of PHS net or the data transmission method that satellite communication network transmits encoded video images/still image, and the coding and decoding device in the data transmission system of this method of employing.

Background technology

Because in recent years to the digital coding of various information such as pictorial information and the development of broadband networks technology, adopt these The Application of Technology to obtain huge development, and developed the system that adopts communication network transmission compressed encoding image etc.

As visual telephone, it is the technology of small data quantity that TeleConference Bridge and Digital Television have adopted video, voice messaging compressed encoding, with the technology of compressed video data stream, compression sound code stream and other data stream multiplexing to one code stream, its technology of transmission/preservation.

To video compression coding, developed motion compensation technique, discrete cosine transform (DCT), sub-band coding, hierarchical coding, the method for technology such as variable-length encoding and fused these technology.The national standard method for video coding comprises ISO MPEG1, MPEG2, and H.263 H.262 ITV-TH.261 reach.The standard method of the compressed bit stream of country's multiplex video, voice/audio signal and other data comprises the ISO mpeg system, and H.223 ITU-TH.221 reaches.

In the conventional video coding as one of above-mentioned video coding national standard method, vision signal is divided into frame, and every frame is divided into the sub-district again, and like this with GOBs, macro block or similar unit come code signal.Every frame adds the header of indication coded system etc. in GOB and the macro block.Whole GOBs or need these headers similarly the time decodes.When the mistake of introducing as if transmission path/storage medium makes Video Decoder can not decode header like this, the whole frame that adds this header, GOB etc. all can not be correctly decoded.Thus, the quality of Video Decoder reconstructed image greatly reduces.

During with communication network transmission compressed encoding pictorial data, receiving terminal must reach the transmission bit stream of " 1 " decoded data with the reconstruct important information from " 0 ".For this reason, as the information of the dedicated rules that indicates coding predetermined block bit stream, above-mentioned header plays a very important role.Such header comprises as the information of the type of prediction that indicates current encoded frame (intraframe coding, interframe encode etc.), indicates the information of frame Displaying timer (time reference), the step-length information of quantification etc.If lost these headers, the information of Chuan Songing can not be correctly decoded thereafter.

Supposing for a certain reason has a mistake in the bit stream, makes the information that will indicate the frame type of prediction change intraframe coding into from interframe encode.At this moment, even actual information is thereafter correctly transmitted, because the decoding end decision signal is intraframe coding, final signal can not be correctly decoded, so the quality of Video Decoder image reconstruction reduces greatly.

To prevail now and use the system of wireless communication networks.Even the error rate of supposing used satellite communication is extremely low, and has set up the system that uses radio-frequency (RF) communication system.Therefore sufficiently do not consider the anti-mistake of transmission code stream structure, do not protect important information such as header well to avoid the mistake of transmission path.

In a part of PHS system of mobile communication main flow, error rate is about 100,000 times or 1,000,000 times of satellite communication.So only according to the error correction of conventional art bitstream encoded, error correction fully.Resemble in the Internet system of PHS system, it is predicted to be the part of communication main flow, and the statistical law of the error type that may occur in the data is not clear, so can not carry out suitable error correction sometimes.In addition, in PHS and the Internet system, may the lost part code stream information.In theory, error correction can not be handled this situation.Therefore, code flow structure itself will have anti-mistake.

As above-mentioned, do not consider the anti-mistake of transmission code stream structure well.Also well important information such as header are not considered the transmission path mistake, particularly, image quality reduced greatly when the transmission path mistake was wherein arranged.

Summary of the invention

An object of the present invention is to provide an information transferring method, by anti-mistake being provided for code flow structure itself, and in adopting the information transmission system of this method, provide a coding/decoding device, even also decoded picture signal in high quality when introducing the transmission path mistake in important information such as the header.

According to the present invention, one information transferring method is provided, comprise: make transmitting terminal after additional reconfiguration information, send the required reconfiguration information of content of reconstruct header or division header information to coded message, make the mistake of receiving terminal check errors header or division header information, when finding mistake in the verification, use reconfiguration information to come decoding and coding information as an alternative.

According to the present invention, one information transferring method is provided, after reconfiguration information is added to data flow, the transmission reconfiguration information is built in the required header of coded message the perhaps content of division header information again with reconstruct, when receiving terminal detects mistake in header or partial information, with reconfiguration information decoded data stream as an alternative.

According to this information transferring method, even header is wrong, and header uses the appointed information specified message can correctly proceed decoding processing as an alternative can not be used to decode the time.Available head information or division header information, the different information between the important information of also available last delivery header information or its part and current transmission, or similarly as reconfiguration information.

In addition, according to the invention provides a kind of information transmission system, wherein information is divided into two or more multi-layered (as the image layer, the GOB layer, macroblock layer, the piece layer), after giving every layer of adding synchronizing signal and header, transmit decoding required synchronizing signal and header, wherein there is a code device look to draw together in header to insert a device that the appointed information of preassigned pattern arranged, reach a device, it can send a information that last layer sent or a part wherein, can send the information that sent with one deck or a part wherein, maybe can send the information of the information content that is used for the reconstruct last layer or sent (image is as human body or face) or wherein a part of, and comprise and the corresponding decoding device of code device with one deck.

According to the present invention, one information transmission system is provided, it sends decoding required synchronizing signal and header after adding synchronizing signal and information, one information transmission system code device wherein is provided, comprise that one will have the insertion portion of the appointed information insert head information of preassigned pattern, and a transmission part that is used to send information or the wherein a part of information that has sent or the information that is used for this information content of reconstruct or a part, and one with the corresponding decoding device of code device.

According to the present invention, one information transmission system is provided, it sends synchronizing signal and this information after adding synchronizing signal and header, one code device wherein is provided, it comprises that one inserts an insertion portion that the appointed information of preassigned pattern arranged in header, and the transmission part of certain information of transmission, this information be used for with the encoding process of header relevant portion from the header relevant portion before the encoding process of part changed and adopted, even wherein code stream itself has anti-mistake so that wrongly in the header also can be correctly decoded, and have one with the corresponding decoding device of code device.

In the said apparatus, preferentially adopt the required information of the decoding of all layers as reconfiguration information, its transmission is indicated by appointed information.In addition, its transmission can be integrated with in the information that the decoding of all layers all needs by decoded information in the reconfiguration information of appointed information indication and another part.In addition, a kind of mode of the available original bit pattern that is defined as header rather than appointed information is as the bit pattern of appointed information.In addition, in the transmission diagram image information, be the information of indication one frame Displaying timer by the specified information priority of appointed information, the information of indication frame type of prediction, the little information of basis in quantification step, or the information of indicator diagram picture frame time can transmit this information.

According to the present invention; comprise one or more image encoders that are used to receive with the compressed encoding picture intelligence in the code device; and comprise the image code stream of multiplexed each image encoder output and the multiplexer of other information code currents; this multiplexer is also exported the multiplexing code stream that comprises a multiplexed head and multiplexed Payload; one information carrying means is wherein arranged; it have one in image code stream or multiplexing code stream the insertion portion of insert head information in the appropriate section; reach an error correction/error detecting code that will produce and be added to multiplexed head from the information the multiplexed head; and with one error correction/error detecting code give with multiplexed head in header in the image code that multiplexed relevant other information send error protection is provided, and send the transmission part of this header and other information.

Be the header in the image code stream and multiplexed the strong error protection that an employing error correction/error detecting code is provided in this code device, in header, after the introducing mistake quality of decoded picture reduced greatly.Therefore, even transmit a compressed image, still can obtain high-quality transmitted image by an insecure transmission path/storage medium.

The header that is included in the image code stream in the multiplexed head can be the information of the time of indicator diagram picture frame.Adopt this information, even be included in the information dropout of time of the indicator diagram picture frame of header in the image code stream, also can from multiplexed header decode go out picture frame the time to information.Therefore, can be at suitable time showing/reconstruct decoded picture, and the correct boundary of picture frame in the detected image code stream.

In addition, according to the present invention, one encoding-multiplexing equipment is provided, it comprises the part charge of a pair of multiple compressed code string segmentation that gets with coding unit compressed encoding input signal, one produces first multiplexing section of multiplexed code element by add filling bit in segment unit segmentation compressed code string, and one is second multiplexing section that the multiplexed element number string of predetermined length integral multiple produces multiplexing code stream by multiplexed its length.

Because in such mode, each multiplexed element number string is that to make its length be the predetermined length integral multiple by add filling bit in segment unit, by the latter end of the compressed code string in each multiplexed element number string is partly compared with the beginning of filling bit, the multiplex machine of decoding/separate can easily be found mistake.In addition, even also can not produce pseudo-synchronous code easily when wrong, can reach high anti-mistake.

In addition, encoding-multiplexing device of the present invention comprises a condensing encoder, it produces the compressed code string this signal is divided into the thing code element of delimiting the organizational structure to input signal coding, reach the multiplexing section that a synchronization code word of collecting no less important from segmentation compressed code string produces multiplexing code stream, in the multiplexing code stream of gained, insert the sign indicating number of the delimiter between the indication coding unit according to corresponding importance.Adopt this processing, can provide error protection to it, therefore when the transmission path mistake is arranged, improved the quality of decoded picture according to each code word importance separately.

Description of drawings

Fig. 1 is the block diagram according to the code device of the information transmission system use of embodiment of the present invention 1;

Fig. 2 is the block diagram according to the decoding device of the information transmission system use of embodiment of the present invention 1;

Fig. 3 A, 3B illustrate according to the embodiment of the present invention a frame is how to be divided into multilayer in 1 the information transmission system;

Gu 4A to 4C illustrates the corresponding bit stream of each layer among Fig. 3 A, the 3B;

Fig. 5 A to 5E illustrates the form of the bit stream that replaces Fig. 4 A to 4C;

Fig. 6 A, Fig. 6 B illustrate the situation that single layer is only arranged according to a frame in the information transmission system of embodiment of the present invention 1;

Fig. 7 is the block diagram according to another code device of information transmission system use of the present invention;

Fig. 8 be with Fig. 7 in the block diagram of the corresponding decoding device of code device;

Fig. 9 A, 9B show according to the internal state of a frame after the information transmission system refresh operation of execution mode 1 and corresponding bits stream;

Figure 10 A, 10B show another example relevant with the content of the important information that is transmitted by the information transmission system according to execution mode 1 and give;

Figure 11 is and Figure 10 A the block diagram of the corresponding decoding processing circuit of 10B;

Figure 12 shows the situation that designates the information as the division header information table that execution mode 1 uses;

Figure 13 has explained district to be encoded in the frame that uses according to the information transmission system of embodiment of the present invention 2;

Figure 14 A to 14D shows the example of the visual code stream of execution mode 2;

Figure 15 shows the timeslice of the information that comprises in the visual code stream of Figure 14 A to 14D;

Figure 16 is the block diagram of the decoding device of execution mode 2 uses;

Figure 17 A to 17C illustration execution mode 2 vop header and the video packet header of using;

Figure 18 is the block diagram of another decoding device of execution mode 2 uses;

Figure 19 is the entire block diagram according to the image/sound encoding device of the information transmission system use of embodiment of the present invention 3;

Figure 20 is the entire block diagram of the image/sound decoding device of execution mode 3 uses;

Figure 21 A and 21B illustration the video code flow that uses of execution mode 3;

Figure 22 illustration the multiplexing code stream that uses of execution mode 3;

Figure 23 A, 23B are multiplexed first examples that execution mode 3 uses;

Figure 24 A, 24B are multiplexed second examples that execution mode 3 uses;

Figure 25 is second example of the multiplexing code stream of execution mode 3 uses;

Figure 26 is the 3rd example of the multiplexing code stream of execution mode 3 uses;

Figure 27 A, 27B are the 3rd examples in the present invention video packet header of using:

Figure 28 A, 28B are the 4th examples in the present invention video packet header of using;

Figure 29 is that storage is based on the medium of information of the present invention and the block diagram of this media decodes device;

Figure 30 is the procedure graph that decoding is stored in the information of Figure 29 medium;

Figure 31 A to 31D shows the bit that adds of the present invention to prevent the situation of pseudo-synchronous code in code stream;

Figure 32 A to 32C illustration the used sign bit of code stream of the present invention;

Figure 33 is the example that the present invention has used the bit stream of a slicing layer;

Each illustration of Figure 34 A to 34C the used video code flow of embodiment of the present invention 4;

Figure 35 A, 35B show the method that execution mode 4 is provided with synchronous code and filling bit;

Figure 36 is the block diagram of the multiplexer of execution mode 4;

Figure 37 illustration adjust the output of layer in the execution mode 4;

Each illustration of Figure 38 A to 38C the output of multiplexed layer in the execution mode 4;

Each is that how video code flow is adjusting layer by first example of minute face in the execution mode for Figure 39 A to 39C;

Each is that how video code flow is adjusting layer by second example of segmentation in the execution mode 4 for Figure 40 A to 40C;

Figure 41 A, each is that how video code flow is adjusting layer by the 3rd example of segmentation in the execution mode 4 for 41B;

Each is that how video code flow is adjusting layer by the 4th example of segmentation in the execution mode 4 for Figure 42 A to 42C;

Each is that how video code flow is adjusting layer by the 5th example of segmentation in the execution mode 4 for Figure 43 A to 43C;

Figure 44 A, each is that how video code flow is adjusting layer by the 6th example of segmentation in the execution mode 4 for 44B;

Figure 45 A, each is that how video code flow is adjusting layer by the 7th example of segmentation in the execution mode 4 for 45B;

Each is that how video code flow is adjusting layer by the 8th example of segmentation in the execution mode 4 for Figure 46 A to 46D;

Figure 47 example has been released in the execution mode and how have been filled on the adjustment layer;

Figure 48 is second example block diagram of multiplexer in the execution mode 4;

Figure 49 illustration the code stream that produces on the FlexMux layer of the multiplexer of structure such as Figure 48 in the execution mode 4;

Another synchronous mark again that Figure 50 has explained a frame in border between addressed location and the execution mode 4;

Figure 51 has explained in the execution mode 4 how to come the differential gap error protection according to the importance in each district in the frame;

Each illustration of Figure 52 A to 52C another form of addressed location in the execution mode 4;

Figure 53 is the circuit block diagram of the encoder of code device among Fig. 1; And

Figure 54 is the circuit block diagram of the decoder of decoding device among Fig. 2.

Embodiment

Embodiments of the present invention are described below with reference to accompanying drawings.

Fig. 1 is the structure of code device according to an embodiment of the present invention.The picture intelligence of camera 101 inputs becomes digital signal by A/D converter 102.This digital signal is sent to encoder 103.This encoder dct transform quantizes, variable-length encoding, and re-quantization, anti-dct transform, motion compensation etc. are carried out efficient compressed encoding to vision signal, and produce an encoded data stream.In this coding, the required important information of decoding is inserted in the data flow.The important header of in the encoded data stream this is input to important header reconfigurable circuit 104 temporarily to preserve it.After the encoder 103 is a Bit String reconfigurable circuit 107.In the circuit 107, the final code stream of transmission path is delivered in decision, promptly according to the data flow of MPEG-2 etc.

In the Bit String reconfigurable circuit 107, a synchronous signal of synchronous signal circuit 105 decisions is added to the head of data flow with predetermined bit stream unit.Thereafter appointed information is inserted circuit 106 appointed information is inserted bit stream.Insert this appointed information operation in the bit stream and allow to add important header to bit stream.Suppose that important header just is inserted into after specifying.From important header reconfigurable circuit 104, extract being added in the bit stream of important header.The back will be described the details of this bit stream structure with reference to figure 4A to 4C and 5A to 5E.

Other coded messages such as final bit stream, voice messaging and character information of multiplexer 108 multiplexed Bit String reconfigurable circuit 107 decisions, and be output to transmission path 110.Notice that encoder 103 outer users can specify arbitrary division header information as important header.

Fig. 2 is the structure of decoded portion of the present invention.

Demultiplexer 121 is separated the bit stream that is sent and is multiplexed as pictorial information, voice messaging, character information etc.122 pairs of pictorial information bit streams of synchronizing indicator carry out synchronous detecting to detect the starting position of bit stream decoding.This information is sent to the decoder 124 that can begin or restart decoding processing.Decoding is from the header of the superiors.Whether error detection circuit 125 these decoded signals of check are wrong.If judgement is wrong, because appropriate section can not be used, corresponding information is sent to important information circuit 126.When the header of one deck was descended in decoding, after synchronizing indicator 122 had been carried out synchronously, identical bit stream was sent to the appointed information decision circuit 123 of check appointed information content.By this operation, can check whether to have added important header.If added this information, then check the type of important header and add the implantation site.According to testing result, appointed information decision circuit 123 outputs one operational order is to decoder 124.Decoder 124 decodings reach the important header that adds on it when the header of anterior layer.The result of important header of decoding be sent to important information circuit 126 with temporary transient preservation it.If receive that from error detection mistake circuit 125 mistake introduces confirmation signal, it shows that the important header of last layer can not use.Therefore the important header sent here when anterior layer of decoder 124 usefulness replaces the important header of last layer, with the code stream that continues to decode thereafter.D/A circuit 127 transfers decoded information such as pictorial information to analog signal to show on display 128.

The image code flow structure that this execution mode uses is described below.

Fig. 3 A, 3B show the notion of the frame that is divided into multilayer.

One frame 200 is divided into a large amount of sections (slice) (macro block line (macroblock line)) 201, wherein gives birth to one and forms (Fig. 3 A) by 16 (point) * 16 (point) macro blocks.Each section 201 is one group of macro block 203 (Fig. 3 B).Frame 200 is on the whole corresponding to the superiors.Each section 201 is corresponding to following one deck.Each macro block 203 is corresponding to following one deck of following one deck.

Fig. 4 A, 4B, the 4C illustration Fig. 3 A, the data structure of corresponding each layer among the 3B.

Fig. 4 A is the bit stream of the superiors among Fig. 3 A.Fig. 4 B is the traditional bit stream corresponding to the slicing layer of cutting into slices among Fig. 3 B.Fig. 4 C be with Fig. 3 B in the section corresponding slicing layer the new suggested bit stream.

Show the image code in the frame layer, i.e. synchronizing signal (the visual opening code of a frame image sign indicating number from showing visual starting position as Fig. 4 A; PSC) beginning.Be indication frame regeneration timing reference (TR) after the PSC, and the type information (PT) of indication predictive coding type such as intraframe coding or interframe encode.Be quantification step size information (PQ) after the PT.TR, PT, these information of PQ are decodings or show that entire frame is necessary.If owing to have mistake etc. to destroy these information, even later layer has been set up synchronously and can not be correctly decoded or show.Low layer information is stored in " PQ " " Data " afterwards.Fig. 4 B is the typical bit stream of " Data ".

Shown in Fig. 4 B, at slicing layer, the visual code stream of each section 201 then is prediction type (SPT) and slice number (SN) from showing synchronizing signal (SSC) beginning of sign indicating number beginning.Be provided with at last and quantize step size information (SQ)." SQ " " Data " afterwards is about being lower than the macroblock layer information of slicing layer.

Below with reference to Fig. 4 C the slicing layer structure that execution mode 1 uses is described.

As above-mentioned, the information among Fig. 4 A is important information.If this information is unavailable,, can not be correctly decoded frame even it is not destroyed to be lower than the slicing layer information of this frame layer.For being correctly decoded the information on the slicing layer that is lower than the frame layer,, need among identification Fig. 4 A and the corresponding header content of slicing layer even when information is unavailable among Fig. 4 A.Therefore in the execution mode 1, prepared to have the sign indicating number of the predetermined bit mode of an indication appointed information at " SPT ".When this yard occurring, header is transmitted once more at slicing layer among Fig. 4 A that has been transmitted.At this moment, transmit " TR ", with " PT " (at this moment, " and SPT ' as appointed information, because of " SPT " do not point out type of prediction, needs " PT ").If the frame layer of Fig. 4 A does not make mistake, do not use these information (TR and PT).If, can continue decoding as an alternative with these information among Fig. 4 C (TR and PT) because the information of frame layers such as mistake is destroyed.

Fig. 5 A to 5E is for replacing another example of Fig. 4 A to 4C.

The same among Fig. 5 A among frame layer and Fig. 4 A, but Fig. 5 B, the appointed information in each slicing layer among the 5C in the insert head information is different from Fig. 4 B, 4C.Fig. 4 B, in each slicing layer shown in the 4C, appointed information is prepared in " SPT ".Fig. 5 B in each slicing layer, inserts a new bit (IS) among the 5C.This bit IS is a bit of representative two class identification informations." IS " can comprise that 2 bits are to represent four class identification informations.

When being the important information of frame layer below " IS " points out,, send afterwards " TR " at " IS " of Fig. 5 C.In decoding end, if, can use " TR " on the slicing layer because the information of frame layers such as mistake is destroyed.At this moment, because " SPT " only represent type of prediction, do not resemble and on slicing layer, transmit " PT " again Fig. 4 A to 4C.

Fig. 5 D is the improvement of Fig. 5 B bit stream.At this moment, slicing layer does not send " SPT ".When specifying the important information of retransmission frame layer according to " IS ", slicing layer needs " TR " and " PT ", shows as Fig. 5 E.

Fig. 6 A, 6B are the situation that a frame only has an individual layer and a bit stream.

At this moment, show that frame is divided into piece (macro block) simply as Fig. 6 A.Show as Fig. 6 B, only come a synchronous frame image code stream with a synchronous signal PSC.At this moment, " TR " and " PT " also are important, if they are destroyed, even information subsequently can not be decoded by correct transmission.Be effective with the mechanism that passes these important informations someway again this moment.Particularly in the random error incident, compare with only sending once these information, all ruined long-pending rate of information TR and PT greatly reduces.In the burst error incident,, also can reduce their ruined possibilities if after sending TR and PT certain hour for the first time, send them again.In the bit stream, " IS " is inserted in information TR shown in Fig. 6 B, and PT is after the PQ etc.According to finger the present of this signal indication, " TR ", " PT " etc. can be inserted into " IS " afterwards.Owing to above reason, the time interval that sends appointed information IS after sending important information is preferably the statistical duration that equals or be longer than burst error.

Fig. 7 is another structure example of coded portion of the present invention.

A/D converter 302 is digital signal with the picture inversion of camera 301 input and exports it to encoder 303.After the encoder 303 is bit stream reconfigurable circuit 307.The final bit stream of transmission path is delivered in 307 decisions of bit stream reconfigurable circuit.When used network easily makes mistake, generally carry out the refresh operation do not estimated so that the long-pending rate that transmission error is not corrected minimizes with predetermined space.Can carry out such refreshing (at this moment, the frame type of prediction is intraframe coding) to entire frame.Yet owing to refresh the amount of information that the amount of information of (intraframe coding) generation produces much larger than interframe encode, this technology is difficult to carry out the low rate encoding transmission.Therefore a kind of technology of preferential employing only refreshes the part of every frame in a plurality of successive frames, thereby has refreshed a frame in multiframe in the required time.In addition, when decoding end detected mistake, it was important from only making wrong part by passing that output passes request again and again again.

For realizing these operations, encoder 303 its type of prediction when coding must be switched in frame and between interframe encode.Suppose and have only predetermined portions that (this moment Fig. 3 A, the special use section shown in the 3B) need refresh, because the type of prediction of this section is different from former section, this information is very important.In addition, very much not be same as interframe encode owing to quantize the step size in refreshing, this information is very important.

In the code device shown in Figure 7, encoder 303 is carried out when refreshing coding, and the required information of encoding is sent to appointed information and inserts circuit 305.Above-mentionedly refresh required important information and be stored in encoding process in advance and change information circuit 306.Bit stream reconfigurable circuit 307 adds synchronous signal circuit 304 determined synchronizing signals on the head of the section bit stream that most refreshes and be encoded.Thereafter, appointed information is inserted circuit 305 appointed information that identification data has been refreshed is inserted in bit stream.At this moment, owing to can add the required above-mentioned important information of decoding refresh pictorial data, required important information is drawn out of it is added to the section bit stream from encoding process change information circuit 306.With reference to figure 9A, 9B will describe the details of this bitstream format.

Multiplexer 308 is multiplexed with the final bit stream of bit stream reconfigurable circuit 307 decision and other coded message such as voice messaging and character information multichannel.Data result is sent to transmission path 310.Notice that encoding process changes information circuit 306 outer users and can specify any information as important information to be added.

Fig. 8 illustration with the corresponding decoding device of Fig. 7 coded portion.Demultiplexer 320 will send code stream and separate and be multiplexed as pictorial information, voice messaging, character information etc.321 pairs of pictorial information bit streams of synchronizing indicator carry out synchronous detecting to detect the decoding starting position of bit stream.When this information is sent to decoder 323, begin or restart to decode.Bit stream also is sent to the appointed information decision circuit 322 of judging the appointed information content.When execution refreshes, because as long as corresponding to this appointed information, according to type of prediction be in the frame or interframe encode to change the decoding type just much of that, come the switch of interframe decoder 325 and interframe decoder 324 in the switching encoding/decoding device 323 according to the output signal of appointed information decision circuit 322.Intraframe decoder device 324 is carried out and is refreshed the decode operation of section.Control intraframe decoder according to above-mentioned important information as quantizing the step size.The pictorial information that D/A circuit 326 decodes the intraframe decoder device 324 in the decoder 323 or interframe decoder 325 is converted to analog signal and is shown on the display 327.

Fig. 9 A, 9B are the form of refresh frame, and the structure of corresponding visual code stream.

Frame 351 is divided into a plurality of sections.Suppose that section 353 is sent out after the section 352 of interframe encode (Fig. 9 A).Next section 354 also is an interframe encode.Fig. 9 B is the bitstream format of frame to be sent this moment.The part 361,362,363 of this bit stream is corresponding to the section 352,353,354 of Fig. 9 A." SPT2 " that be used to refresh in the section bit stream locates to insert the appointed information that indication refreshes with intraframe coding.Information subsequently " SQ2 " shows the quantification step size that refreshes usefulness.All " Data2 " are all decoded in the mode of intraframe coding.

Figure 10 A, 10B are another example of the important information content.Figure 10 A is the bit stream of frame layer.Figure 10 B is the bit stream of slicing layer.

With reference to figure 4A to 4C, be " TR " this information representation Displaying timer after the appointed information SPT on the slicing layer, but its bit number can be very big sometimes.For preventing the situation of this inconvenience, shown in Figure 10 A, the 10B in, adopted a technology that given difference with the corresponding information that has sent is encoded.This technology generally is used for compressed encoding.

In more detail, if TR is 0 to 255, then need 8 bits to represent its value.Yet supposition can be set up a condition.The feasible low-speed range value that does not have corresponding to three frames or more frame.At this moment, because contiguous frames can not be separated from each other by three frames or more frame in showing, if relative time is just much of that with regard to (the low-speed range value is 0,1,2,3) much of that with reference to representing 4 states.At this moment, " TR " only needs 2 bits.Therefore can reduce bit number.At this moment, because last decoded information, this part can not only decide with " TR ".

Figure 10 B is with above-mentioned difference TR (DTR among Figure 10 B) when making important header, bit stream example to be sent on the slicing layer.Decoding " DTR ' after, on TR information in the bit stream that " DTR " is added to former frame decoded on Figure 10 A frame layer.Just can calculate the TR of present frame.

Figure 11 is at Figure 10 A, and the situation of 10B is carried out the circuit of decoding.Circuit among Figure 11 replaces the decoded portion among Fig. 2.At first, synchronous detecting is thought highly of 22 decodings that detect the bit stream that demultiplexers 121 send here and is begun part, decoder 401 decoded frame layers " TR,,, " PT " etc.Decoded information TR is stored in the memory (2) 404 simultaneously.Whether error detection circuit 402 detects in the information header wrong.The result delivers to and deposits mansion device (1) 403 (above-mentioned processing is used for the bit stream of Figure 10 A).To the slicing layer of Figure 10 B, at first, 123 decisions " SPT " of appointed information decision circuit.Decoder 401 decodings then " DTR ' with " PT '." DTR " is sent to memory 403.If find from the information that error detection circuit 402 is sent here, because mistake etc. can not be used the header (TR) of previous frame layer, memory 403 outputs one ask to be sent to memory 403 to memory 404 with the information TR of the former frame that will be stored in memory 404.In the memory 403, this information TR and the information D TR of above-mentioned present frame are formed " TR " of present frame mutually.This information is recycled to decoder 401 to continue decoding.Information TR also is sent to memory 404 to treat that same treatment is used in the next frame.

Figure 12 is the bit mode of preparing for other header that is not used as appointed information.

Suppose that preferential give " SPT " distributes 2 bit modes.At this moment, three kinds of type of prediction are arranged, I (intraframe coding), P (forward predictive coded), B (bi-directional predictive coding) has distributed 00,01,10 bit sample values respectively." information of 11 ' correspondence, this sign indicating number is as appointed information owing to use.Even " SPT " is " 11 ", and it does not represent any type of prediction but explanation has important information thereafter.Note important information that appointed information is pointed out can be header (TR, PT, PQ) or its part.Alternatively, this information can comprise data (as " DATA " of Fig. 4 A) subsequently.Can be according to system request, the network error frequency of occurrences, required encoding rates etc. change these information.

As above-mentioned, according to execution mode 1, as if the important information of having lost as header, the information of the instruction adding/transmission reconstruct important information of appointed information according to the rules.Therefore, even important header is wrong, this information can not be rebuilt, can continue to be correctly decoded by using the appointed information that transmits subsequently and the reconfiguration information of appointed information appointment.

Embodiments of the present invention 2 are described below.

Roughly the code device with Fig. 1 is identical according to the overall structure of the code device of execution mode 2.In the execution mode 2, every frame (also being an image or VOP) also is encoded so that be divided into the sub-district (sub-district fences up with dotted line among Figure 13) that is called macro block.In addition, when in the visual code stream of coding image signal when wrong, for the code stream that allows every frame recovers synchronously, this frame is encoded with the video packets unit (impaling with solid line among Figure 13) that consists of a plurality of macro blocks.

Figure 14 A to 14D is the example of the visual code stream of code device output.Figure 14 A is the whole visual code stream (VOP) of a frame.In vop header, add the VOP opening code as synchronization code word (among Figure 14 A " VSC ") that independently to decode, and comprise vop header (among Figure 14 A " vop header ") about the header of VOP.

The VOP code stream is divided into the video packets code stream that comprises macro block (among Figure 14 A " MB data ') again.In the visual code stream head of each video packets, add the synchronization code word (" RM " among Figure 14 A) can independently decode and be called as again synchronous mark, and the video packet header after the synchronous mark (among Figure 14 A " video packet header) again.Yet note to add again synchronous mark (RM) and video packet header (video packet header) in first video packets owing to be added with VOP opening code and vop header in first video packets of VOP.

Figure 14 B illustration the header in the vop header.With reference to figure 14B, mould time base (MTB) and VOP incremental time (VTI) are the temporal informations of VOP.These information are used for the decoding and the display timing generator of VOP frame.

Below with reference to the time that Figure 15 describes VOP, the relation between mould time base (MTB) and VOP incremental time (VT1).The VOP incremental time is with the accuracy representing VOP time of millisecond, and its value is the remainder of VOP time divided by 1000 milliseconds of (1 second) gained.Mould time base is the time of accuracy representing VOP with the second, and then it is " 0 " if the time of VOP is identical with the second of last coding VOP.If number difference second of VOP, mould time base is a difference.

As the VOP time (millisecond) be 0,33,700,1000,1300,1833,2,067, then the VOP incremental time is respectively the remainders that remove these values with 1000, promptly 0,33,700,0,300,833,67.If the time divided by 1000 and the value of removing all numerals behind the decimal point (among Figure 15 0,0,0,1,1,1, or 2) gained be different from last VOP then value be 1.Promptly corresponding to time=1, the mould time of 1000,2067 VOP base is " 1 ", and its value of the VOP of remainder is " 0 ".In addition, the available variable length code mould time base of encoding.As mould time base is 0,1,2 ..., variable length code is " 1 ", can set " 01 " according to mould time base, " 001 " ....

The VOP prediction mode (VPT) of Figure 14 B is the information of the whole frame predictive coding mode (I, B or P) of indication.VOP quantization parameter (among Figure 14 B " PQ ") is indication be used to the to encode information of quantization step of VOP.Yet when being divided into a plurality of video packets VOP being encoded, because quantization step changes with the unit of video packets, this information can be used as the information of the quantization step of pointing out first video packets.

Figure 14 C, the 14D illustration be added to information in video packet header of a video packets as header.Macro block number (scheming inferior 4C, among Figure 14 D " MBA ") is the information of expression video packets first macro block number.The video packets quantization parameter (Figure 14 C, and among the 14D. " SQ ') be the information of the quantization step of expression video packets.Header expansion code (Figure 14 C is among the 14D " HEC ", corresponding to " IS " among Fig. 5 B to 5E) is whether an expression has added the sign of share the important information of (multiplexed) with video packet header." HEC " " 0 ' time, do not add important information, shows as Figure 14 C." HEC " is " 1 ", the time, added important information, show as Figure 14 D.In Figure 14 D situation, be can the restructuring graph picture frame temporal information, the mould time base (among Figure 14 D " MTB ") of expression VOP time and VOP incremental time (among Figure 14 D " VTI ,) addition, and as important information by adding with revising.

Figure 16 is the block diagram of device of the visual code stream of decoding Figure 14 A to 14D.Identical label is represented the part identical with decoding device among Fig. 2 among Figure 16, only describes both differences below.The structure of decoding device, decoding device comprises a vop header decoder 601, one video packets decoders 602, one a time decoder 603 and a buffer storage 621 among Figure 16 in Fig. 2.

After detecting the VOP opening code, synchronous detecting is thought highly of the signal that 124 1 expressions of 22 notice decoders have detected sign indicating number.This notice is responded, and decoder 124 sends a code stream that comprises a vop header and VOP opening code, and promptly first video packets is to the vop header decoding 601 of the vop header of decoding.Vop header decoder 601 decodes the temporal information in the vop header, VOP coded system information, VOP quantization parameter.In these information, temporal information is the temporal decoder 603 that mould time base and VOP incremental time are sent to decode time information.

Mould time base and VOP incremental time that temporal decoder 603 decodings are sent here, and whether wrongly detect.Whether can temporal decoder 603 be checked wrong for a real time by the time that check is basic from the mould time and VOP time incremental time decodes out.When being the NTSC signal as coding image signal, because frame per second is 30Hz, the time the should be 1/30 second integral multiple of (=33 milliseconds).Therefore if decode time is not 1/30 second a integral multiple, then mould time base and VOP incremental time have the transmission path mistake.When coding image signal was the PAL signal, whether was 1/25 second integral multiple its Check-Out Time.

As the reference value of error detection, a predetermined value can be set in the coding/decoding device according to picture intelligence type (PAL, NTSC, CIF or similar), or the information of expression reference value can be inserted in system information code stream (not shown) or the partial images code stream.

After temporal decoder 603 has been finished the decoding and error detection of temporal information, then send an expression vicious signal to vop header decoder 601 if judgement is wrong.If judge inerrancy, then send the signal of an expression decode time information to vop header decoder 601.During the temporal information inerrancy, vop header decoder 601 is stored in buffer storage 621 with this temporal information, and this information is delivered to decoder 124 with out of Memory.If temporal information is wrong, abandon the first video packets sign indicating number that comprises vop header, next video packets begins to decode.

After having detected again synchronous mark (RM), the signal that synchronizing indicator witch 22 send an indicator sign to detect to decoder 124.This signal is responded, and decoder 124 send a code stream that comprises the video packet header that is right after again synchronous mark to the vop header decoder 601 in decoded video packet header, and promptly second or video packets subsequently.Macro block number (MBA), video packets quantization parameter (SQ) and header expansion code (HEC) in the video packets header decode device 602 decoded video packet header.

If header expansion code (HEC)=" 1 ", the temporal decoder 603 that mould time base subsequently and VOP incremental time are sent to decode time information.As the situation of last vop header decoding, mould time base and VOP incremental time that temporal decoder 603 decodings are sent here, and whether check is wrong.After temporal decoder 603 has been finished temporal information decoding and error detection,, then send the vicious signal of an expression to video packets header decode device 602 if judgement is wrong.If judgement is error-free, then send the signal of an expression decode time information to video packets header decode device 602.If temporal information is wrong, video packets header decode device 602 abandons the video packets code stream that comprises vop header, and next video packets of decoding.

If the temporal information inerrancy, with temporal information be stored in first buffer storage 621, promptly the temporal information that obtains from last encoded video bag is compared, to judge the VOP in the current video bag.If their unanimities judge that video is included among the VOP identical with including last encoded video bag, and send the signal of an expression decoder 124 decoded video package informatins, carry out decoding subsequently.In contrast, if decode time information is different from the temporal information that is stored in the buffer storage 621, it judges that video to be decoded is included among the VOP that is different from the VOP that includes last encoded video bag.At this moment, decode time information is stored in the buffer storage 621.In addition, send one to represent between last video packets and this video packets one VOP district is arranged, and this video packets is decoded as the VOP division signals of a new VOP, decode time information, and the signal of expression decoded video header packet information to decoder 124.After having received the VOP division signals, decoder 124 is judged last video packets that last encoded video bag is VOP, and carries out VOP decoding end process.Decoder 124 also judges to be first video packets of next VOP from video packets to be decoded now, and continues the decoded video bag.

Use this processing,, therefore improve the quality of decoded picture even, can determine the VOP border according to the temporal information of video packets, and obtain correct decode time because mistake has been lost VOP opening code and vop header.

Figure 17 A to 17C is second example in vop header and video packet header, and the difference of they and Figure 14 A to 14D is to have added in the check head whether vicious CRC check bit of information.

Figure 17 A is a vop header.With reference to figure 17A, " CW1 " is to the mould time base in the vop header, the VOP incremental time, and the VOP coded system, the VOP quantization parameter carries out the check bit of CRC check.

Figure 17 B, 17C are video packet header.With reference to figure 17B, " CW2 " is to macro block number, the video packets quantization parameter, and header expansion code carries out the check bit of CRC check.With reference to figure 17C, the video packets of " HEC=1 " is only arranged, promptly having added in the video packets of important information just has " CW3 ", and it is to be the check bit that mould time base and VOP incremental time carry out the CRC school to important information.

Figure 18 is the decoding device at visual code stream among Figure 17 A to 17C.Identical label is represented the part identical with Figure 16 among Figure 18.Difference among this device and Figure 16 is to have added CRC decision circuit 605.This difference is only described below.

Information in the vop header decoder 601 decoding vop headers, and vop header is carried out CRC check with CRC check bit CW1.If it is wrong that CRC check is judged, abandon vop header and video packets wherein, and next video packets that begins to decode.

Information in the video packets header decode device 602 decoded video packet header, and video look head is carried out CRC check with CRC check bit CW2.If it is wrong that CRC check is judged, then abandon video packet header and in video packets, and next video packets that begins to decode.If judge inerrancy, and decoding header expansion code " HEC " be " 1 ", and " HEC " the added afterwards important information of then decoding (MTB, VTI).Whether the CRC check bit is used to check important information wrong.If judge inerrancy, then as decoding device among Figure 16, then with these information in vop header and another video packets and temporal information time ratio, and execution VOP segment processing etc.

As above-mentioned, in the execution mode 2,,, also can decode the correct time based on the important information in the video packet header even lost the temporal information in the vop header because of mistake because each video packets has added the information of presentation image frame time in the frame.Therefore decoding device can be with correct time reconstruct and displayed image.

In addition, owing to be by temporal information in temporal information in the video packet header and vop header or another video packet header recently being judged the VOP border mutually,, also can being correctly decoded the VOP border even lost the VOP opening code because of mistake.So improved the quality of decoded picture.

In the above-mentioned example, whether the header expansion code in each video packets (HEC) expression comprises important information.As " HEC "=" 1 ' can be provided with to all video packets adding important information, or only some video packets is set HEC=" 1 ".By control the number of the video packets that adds important information according to the mistake of transmission path, can only protect important information effectively with the unnecessary head of sub-fraction.

Suppose that decode time is by (VTI represents with the VOP incremental time with patrix time base (MTB).At this moment, during MTB=0, even can not be correctly decoded temporal information, the error of decode time is 1 second or still less.Yet if be not correctly decoded " MTB ' for the VOP of " 0 " " MTB ', the decode time error of back VOP is several seconds.So to the VOP of MTB=0, its all video packets all can be made as HEC=0, or only establish sub-fraction video packets HEC=1, and MTB is not made as HEC=1 for all or most of video look among the VOP of " 0 ", is correctly decoded " MTB " thus.

In the execution mode 2, use the information (mould time base and VOP incremental time) of express time to be used as and the two multiplexed important informations in video packet header.Except these information, video packet header can with the information of presentation code mode, the information information relevant of expression quantization parameter with motion compensation, motion vector information is pair multiplexed.

Suppose VOP when coding with VOP be unit switch the VOP prediction mode (as infra-frame prediction VOP (I-VOP), forward prediction VOP (P-VOP), bi-directional predicted VOP (B-VOP).At this moment, if can not be correctly decoded VOP prediction mode information, corresponding VOP then can not decode.When VOP prediction mode information also in video packet header by two when multiplexed, even lost VOP prediction mode information in the vop header because of mistake, also can be based on the VOP prediction mode information decoding VOP in two multiplexed informations in video packet header.This situation is described below.

Figure 27 A, 27B are the 3rd example in the video packet header of execution mode 2.The visual code stream of whole frame and vop header (VOP) and Figure 14 A are identical among the 14B.Figure 27 A, 27B represented respectively head expansion HEC=" 1 ' with video packet header of HEC=" 0 ".Different is for these video packet header and Figure 14 A to 14D, HEC=" 1 ' time except that the information of express time (among Figure 27 B " MTB " with " VTI ') also comprise VOP prediction mode information (among Figure 27 B " VPT ").

Figure 27 A, same Figure 16 of overall structure of the stream decoder of image code shown in the 27B.Yet the work of video packets header decode device 602 is different from Figure 16.In addition, this device is not both with Figure 16's, and VOP prediction mode information (VPT) is stored in buffer storage 621 with temporal information (mould time base and VOP incremental time).To describe the work of decoder below, mainly be the work about difference.

After detecting the VOP opening code, synchronous detecting is thought highly of 22 and is sent an expression to detect the signal of this yard to decoder 124.This signal is responded, and decoder 124 send a code stream that comprises vop header after the VOP opening code to vop header decoder 601, with the decoding vop header.Temporal information (MTB and VTI) in the vop header decoder 601 decoding vop headers, VOP coded system information (VPT), and VOP quantization parameter (PQ).Mould time base (MTB) in these message and VOP incremental time (VTI) are sent to the temporal decoder 603 of decode time information.

Mould time base and VOP incremental time that temporal decoder 603 decodings are sent here, and whether check is wrong.By check from time that mould time base and VOP incremental time decode out whether be check the real time whether wrong.As coding image is the NTSC signal, because frame per second is 30Hz, temporal information should be the multiple of 1/30 second (=33 milliseconds).Therefore if decode time is not 1/30 second a multiple.It judges that mould time base and VOP incremental time have the transmission path mistake.If code signal is the PAL signal, whether be that 1/25 second multiple carries out error detection by detected signal information.

As the reference value of this error detection, can (PAL, NTSC CIF) set predetermined value in coding and decoding device, or insert the information of expression reference value in system information code stream (not shown) or partial images code stream according to the picture intelligence type.

After time decoder 603 has been finished temporal information decoding and error detection, if it judges wrong signal from mistake to 601 transmissions one of vop header decoder that then express.If it judges inerrancy, send the signal of an express time information to vop header decoder 601.During the temporal information inerrancy, the vop header decoder 601 VOP prediction mode information (VPT) of also decoding.If the also inerrancy of VOP prediction mode information, temporal information and VOP prediction mode information are stored in buffer storage 621, and are sent to decoder 124 with out of Memory.If VOP prediction mode information is wrong in temporal information or the vop header, abandon the code stream of the video packets that comprises vop header, and next video packets of decoding.

After detecting synchronous mark, synchronous detecting is thought highly of 22 and is sent the signal that an expression has detected sign to decoder 124 again.This signal is responded, and decoder 124 sends to the vop header decoder 601 in decoded video packet header and comprises that the back is with the code stream in the video packet header of synchronous mark again.Macro block in the vop header decoder 601 decoded video packet header number, video packets quantization parameter, and header expansion code.

If header expansion code HEC=" 1 ", " HEC " mould time base and VOP incremental time decoder 603 of being sent to decode time information afterwards, decoding with last vop header is the same, mould time base and VOP incremental time that temporal decoder 603 decodings are sent here, and whether check is wrong, after temporal decoder 603 has been finished temporal information decoding and error detection, then send one to express wrong signal to video packets header decode device 602 if its judgement is wrong.If judge inerrancy, send the signal of an expression decode time information to video packets header decode device 602.If temporal information is wrong, video packets header decode device 602 abandons the video packets code stream that comprises this video packet header, and next video packets of decoding.

If the VOP prediction mode information after the temporal information inerrancy, decode time information.If the also inerrancy of VOP prediction mode information is compared the VOP that comprises video packets with judgement with decode time information with the temporal information in being stored in buffer storage 621.If their unanimities, judge that then this video packets and last video are included in same VOP frequently, and send the signal of representing the decoded video header, an an ancient drinking vessel chicken video packets thus to decoder 124.In contrast, if this temporal information is different from the temporal information that is stored in the buffer storage 621, judge that then video packets to be decoded from now on is among the VOP different with the VOP that comprises last video packets.At this moment, decode time information and VOP prediction mode information are remembered in buffer storage 621, and to send these video packets of expression to decoder 124 are VOP block signals of first video packets of VOP, the signal of expression decode time information, and the signal of expression decoded video header packet information.After receiving the VOP block signal, decoder 124 judges that last decoded video bag is the last bag of VOP, and carries out VOP decoding end process.Decoder 124 judges that also video packets to be decoded from now on is first video packets of next VOP, and carries out VOP decoding beginning processing, continues next video packets of decoding thus.

If the VOP prediction mode information in the video look head is different from the VOP prediction mode information that is stored in the buffer storage 621, then use the VOP prediction mode information decoding video packets in the video packet header.Adopt this operation, even can not be correctly decoded the VOP prediction mode information in the vop header, also can the decoded video bag.

Use this processing,,, improve the quality of decoded picture owing to can correctly identify VOP border and VOP prediction mode from temporal information the video packet header and VOP prediction mode even lost VOP opening code and vop header because of mistake.

When supposing decoding vop header or video packet header, error check information (CRC has appearred in the image code stream, the packing ratio top grade), or receive the circuit of code stream from transmission path/storage medium, maybe the code stream that receives is separated the demultiplexer that is multiplexed as visual code stream, speech code stream etc. and whether wrongly can judge in the code stream.At this moment, can whether wrongly determine in decoding vop header or the video packet header with the result of determination that this error detection information and sort circuit obtain.If judge that by this mode these decoded informations are wrong, image decoding does not just use these information yet.Alternatively, abandon the video packets that includes the information of making mistake and not to its decoding.

Figure 28 A, 28B are the 4th examples in video packet header in the execution mode 2.Visual code stream and vop header and Figure 14 A of whole frame (VOP), similar among the 17A.Figure 28 A, 28B represent HEC=" 0 " and HEC=" 1 ' video packet header respectively.These video packet header are different from Figure 17 B, and the 17C part is that HEC=" 1 ' time, removes the outer VOP prediction mode (among Figure 28 B " VPT ") in addition of information (" MTB " reach " VTI " among Figure 28 B) of express time.

At Figure 28 A, the overall structure of the decoding device of visual code stream is identical with Figure 18 among the 28B.Yet the work of video packets header decode device 602 is different among Figure 18.In addition, difference in Figure 18 of this device is VOP prediction mode information (VPT) and temporal information ' (mould time base and VOP incremental time) be stored in buffer storage 621.The work of decoder is described below, and mainly is the work about difference.

Information in the vop header decoder 601 decoding vop headers, and vop header is carried out CRC check with CRC check bit CW1.If it is wrong that CRC check is judged, abandon vop header and comprise the video packets of vop header, and next video packets that begins to decode.

Information in the video packets header decode device 602 decoded video bags, and CRC check is carried out in video packet header with CRC check bit CW2.If it is wrong that CRC check is judged, then abandon video packet header and comprise the video packets in video packet header, and next video packets that begins to decode.If judge inerrancy, and decoding header expansion code HEC be " 1 ", " HEC " afterwards the two multiplexed important information of decoding (among Figure 28 B " MTB ', " VTI ', " VPT ').Whether CRC check bit CW3 is used for the two multiplexed important informations of check wrong.If inerrancy, then as decoding device among Figure 16, relatively these important informations and the time information in vop header and another video packet header is carried out segment processing etc.

As above-mentioned,,, also can decode correct temporal information from the important information the video packet header even lost the temporal information among the VOP because of mistake because the information of express time is arranged in the important information.Therefore decoding device can be with correct time reconstruct and displayed image.In addition, because the temporal information in the video packet header is recently carried out the VOP edge determination mutually with the temporal information in vop header or another video packet header, even lost the VOP opening code, also can be correctly decoded out the VOP border, and improve the quality of decoded picture because of mistake.

In addition, when VOP prediction mode information also is included in video packet header as two multiplexed informations, even because of mistake has been lost the VOP prediction mode information in the vop header, also can be based on being included in the VOP prediction mode information decoding VOP that looks in the step packet header as two multiplexed informations.

In the execution mode 2,, must add the bit that is flag bit for avoiding in vop header and video packet header producing the pattern with synchronous code (visual opening code, VOP opening code, synchronous mark etc. again).

How Figure 31 A to 31D illustration adds flag bit in the code stream of Figure 14 A to 14D.With reference to figure 31A to 31D, VOP incremental time (VTI) " sign " afterwards is the flag bit that predetermined bit value (as " 1 ") is arranged.

Figure 32 A to 32C is the video packet header and the comparison that the video packet header of a flag bit is arranged of no any flag bit.Suppose that the synchronous mark again as synchronous code is that pattern is 17 bit codewords of " 00000000000000001 ", VOP incremental time (VT1) is to be worth 10 bit codewords arbitrarily, " MTB ' be that last position is " 0 ' variable length code.

If flag bit is not set, " VTI ' be continuous " 0 ", then shown in Figure 32 B, produced and the consistent bit mode of synchronous mark again.In Figure 32 B institute example, " 0 " of MTB, " 0000000000 " of " VT1 " has been formed and the same pattern of synchronous mark again with bit stream " 000001 " subsequently.

In contrast, among Figure 32 C, by " VTI ' add flag bit " 1 " afterwards, in the video packet header continuously zero maximum number be 11 (" MTB ' last position " 0 ', " 0000000000 " of " with VTI ").Therefore can not produce and the consistent bit mode of synchronous mark again.

Care mark is than peculiar predetermined value (being " 1 " among Figure 32 A to 32C).Therefore, but whether whether decoding device check mark bit has predetermined value to judge in vop header and the video packet header wrong.

Flag bit also can add to Figure 17 A to 17C in the

execution mode

2,27A, and 27B, 28A is among the 28B.

This code flow structure also can be used for using the code stream of slicing layer.Figure 33 is another code stream that uses slicing layer in the execution mode 1.

With reference to Figure 33, reference symbol SSC refers to the synchronous code of cutting into slices; Value for " 1 ' EPB is added into avoiding in the code stream and is not that the part of synchronous code (as SSC) has identical bit mode, MBA with synchronous code, the expression respective slice in first macro block number information; SQUATNT, the quantization parameter that section is used; GFID, the information of information or its partial information in the presentation image head.When code stream designated bit place is provided with synchronous code SSC, " SSC,, filling bit SSTUF is set before." macro block data " is the data of each macro block.

" TR " temporal information (temporary transient reference) is two multiplexed important informations." TRI ' be to indicate 1 bit identification that whether has added " TR ".During TRI=1, added " TR ".

Embodiments of the present invention 3 are described below.

Figure 19 is the overall structure according to the video/speech coder of embodiment of the present invention 3.The vision signal 101A of coding to be compressed and voice signal 102A are input to video encoder 111A and speech coder 112A respectively, and they are compressed video and voice signal and output video code stream 121A and speech code stream 122A respectively.Owing to describe video encoder and speech coder in detail in the reference (Hiroshi Yasuda, " multimedia coding international standard ", Maruzen (1994)) etc., omit its detailed description.

Multiplexer 130A multiplex video code stream 121A, speech code stream 122A and a data code flow.The result exports a multiplexing code stream 135A.

Figure 20 is the entire block diagram corresponding to the video/speech coder of video/speech coder among Figure 19.Demultiplexer 180A separates the multiplexing code stream 185A that multiplex video/speech coder comes out.The result exports a video code flow 171A, a speech code stream 172A, a data code flow 173A.Video code flow 171A and speech code stream 172A input to Video Decoder 161A and a Voice decoder 162A respectively to decode.As a result, the vision signal 151A of output reconstruct and the voice signal 152A of reconstruct.

Figure 21 A, 21B are two examples of the straight 21A of video code flow.Video encoder 111A with image cell (frame, or VOP) code signal to produce video code flow 121A.Each image is divided into again and is called the macroblock encoding sub-district.

The video code flow of I image is from visual opening code (PSC) 201A (also being the VOP opening code), also can independently be decoded in the starting position of visual opening code presentation image.

After the image opening code 201A image head (PH) 202A (also being vop header).An image 202A comprises PTR (the interim reference of the image) 221A of presentation image temporary position, represents picture coding mode (PCM) 222A of whole picture coding mode, and image quantization step-length (PQ) 223A.After the image 202A is each macroblock encoding data 203A.

Figure 21 B shows is to be the situation of encoding in the unit with the section that consists of a plurality of macro blocks.

The code stream of each section comprises an expression section starting position and synchronous mark again (RM) 210A that can independently be decoded." RM " is macro block data (MB) 203A of section head (SH) 211A and each macro block afterwards.A section 211A comprise first macro block in the expression section number SMBN (section macro block number) 231A and quantization step (WQ) 232A.

Can add again a synchronous mark 210A and a section 211A to the bit of each predetermined number in the picture frame or in the pre-position.When encoding this piece cutting structure,, also can set up synchronous again with the 210A of synchronous mark again that can independently be decoded even wrong in the video code flow.Owing to avoided mistake to reach outside the section, improved the quality of reconstructed image when the transmission path mistake is arranged.

Figure 22 illustration the multiplexed multiplexing code stream 135A of multiplexer.Multiplexing code stream 135A is made up of a plurality of multiplexed bags, each multiplexed video code flow (video) that is multiplexed into pre-sizing respectively that is surrounded by wherein, voice (audio frequency) code streams (voice) and data/control information code stream (data).With reference to Figure 22,301A, 302A, the 303A part is respectively multiplexed bag.All multiplexed bags can be the fixed length bags of length identical (bit number), or the different elongated bag of length.

Added in the head of each multiplexed bag and represented multiplexed multiplexed opening code (MSC) 310A that unwraps the beginning position.After the multiplexed opening code 310A multiplexed head (MH) 311A, 312A, or 313A.After the multiplexed head is to be video code flow 121A among the multiplexed Figure 19 in unit with the bag, the multiplexed Payload that speech code stream 122A, data code flow 103A obtain (321A among Figure 22,322A, or 323A).

Figure 23 A, 23B are first examples of information among multiplexed head (MH) 311A.With reference to figure 23A, 23B, multiplexing code (MC) 351A are expression video code flows (video), speech code stream (voice), data code flow (data) how in multiplexed Payload 321A by multiplexed information.If in this multiplexing code information (MC) the transmission path mistake is arranged, can not identify the multiplex mode of code stream.Therefore, demultiplexer 180A can not be correctly decoded video code flow, speech code stream, data code flow.So Video Decoder 161A and Voice decoder 162A can not be correctly decoded, cause the video of reconstruct and the quality of voice signal to descend.

For avoiding this situation, strong error protection is provided for each multiplexed head (MH) with an error detection code and error correcting code.With reference to figure 32A, 32B, label 343A (CRC) refers to the CRC detection bits; 354A (FEC) error correcting code check bit.

In the execution mode 3, comprise that the multiplexed head (MH) in the multiplexed bag of video code flow (video) comprises video heads information (VHD) 352A and multiplexing code information (MC).Figure 23 A, in the situation, " MH1 " (311A) reaches " MH2 " is the multiplexed head that comprises video heads information (VHD) 352A (312A) shown in the 23B.Video heads information (VHD) 352A is the important information of the coded system etc. of whole image (frame) in the expression video coding.If this information is wrong, the reconstruct boudoir resembles quality and reduces greatly.For example as if video code flow form such as Figure 21 A, shown in the 21B, an image 202A or a section 211A or its partial information are used as video heads information (VHD) 352A and are arranged on multiplexed head.

Execution mode 3 is characterised in that the visual head that has inserted by this way in important information such as the video coding in multiplexed head, and has produced multiplexing code (MC), and error detection code and error correcting code provide strong error protection to utilize these yards.Adopt this feature, and do not provide the conventional video code device of error protection to compare, improved anti-transmission path mistake important information.

Figure 24 A, 24B are second examples of multiplexed head (MH).Figure 24 A, same numeral is represented Figure 23 among the 24B, and the appropriate section of first example among the 23B is only described difference below.What second example was different from first example is except that video heads information (VHD) 352A, and what comprise video code flow (video) multiplexedly is surrounded by an expression video code flow image or a slice boundaries position and is included in graphics pointer (ALP) 451A in the multiplexed head.

If graphics pointer (ALP) 451A is not set, after demultiplexer 180A separates the multiplex video code stream, Video Decoder 161A fibrous root according to visual opening code or again synchronous mark detect image or slice boundaries.In contrast, if comprise graphics pointer (ALP) 451A in the multiplexed head, available this pointer detects image or slice boundaries.Because the graphics pointer in the multiplexed head has strong error correcting capability, has increased the possibility of correct detection image or slice boundaries, and has improved the quality of reconstructed image.

In addition, video heads information (VHD) 352A can comprise all or part information in image or the section head.

Figure 25 shows a situation, except that corresponding multiplexing code (MC1 and MC2) 611A and 621A, in the multiplexed head of multiplexed bag 601A that comprises video code flow (video) and 601B, visual time reference (PTR1 and PTR2) 612A and 622A have only been comprised as video heads information.

With reference to Figure 25, the multiplexed Payload of multiplexed bag 601A includes a section (section N) at the visual code stream tail of PTR=1, visual opening code (PSC) 614A of the image of PTR=2 subsequently, the visual time reference (PTR2) of PTR=2, picture coding mode (PCM2) 616A, and the first half of first section of the image code elephant of PTR=2 (section 1) 617A.The Payload of multiplexed bag 602A comprises first latter half of cutting into slices (section 1) of the visual code stream of PTR=2, synchronous mark again (RM) 624A of second section, section head (SH2) 625A, and second section (section 2) 626A of the visual code stream of PTR=2.

The multiplexed head (MH1) of multiplexed bag 601A has the PTR612A of the image of the PTR=1 that comprises the decline code stream in multiplexed bag 601A.Use an error correcting code and an error detection code (CRC and FEC), for PTR612A and multiplexing code MC1 provide error protection.Even so because of mistake can not be correctly decoded PTR (615A) in the video code flow of multiplexed Payload, owing to can be correctly decoded the PTR (612A) in the multiplexed head, thus can obtain correct PTR, with correct time showing decoded picture.

Use in the method for video coding of piece cutting structure, if in the video heads information of the multiplexed bag that comprises a section opening code (synchronous mark again) and all head, insert a PTR, even, also can judge image border based on PTR because of mistake can not be correctly decoded visual opening code.Suppose and lost visual opening code (PSC) 614A or PART615A.At this moment, the PTR (as PTR612A) among the PTR622A in the multiplexed head of next multiplexed bag and last multiplexed bag multiplexed is compared.If they are inconsistent, then judging among the multiplexed bag 601A has image border.At this moment, can be from first section (among Figure 25 RM624A " the section 2 ") coding that beginning is correct that synchronous mark is arranged among multiplexed bag 602A again.

(as using the coding method of B image) can comprise picture coding mode information in the multiplexed head when having adopted the coding method of the frequent change of picture coding mode.

Figure 26 is the 3rd example of multiplexing code stream.In this multiplexing code stream; each multiplexed bag 701A; 702A and 703A are provided with an image or a section, are image head (PH1) 712A among multiplexed 751A and the 752B and section head (SH2) 722A, and multiplexing code (MC1 and MC2) 711A and 721A provide error protection.If prepare image or section in couples,, do not need from separate multiplexed visual code stream to detect a visual opening code and synchronous mark again and again because each multiplexed bag always is positioned at the starting position of an image or a section as video codes and multiplexed bag.The result has reduced processing quantity.In addition, use to the transmission path mistake have high repellence sign indicating number can reduce and can not right area divide image or section starting position and can not decoded picture or the possibility of section.

In the execution mode 3, encoding and decoding one image/slice signal.Yet the present invention can be equally applicable to use a plurality of images/a plurality of image/voice signals of voice signal encoder encoding-multiplexing, and a plurality of images/voice signal decoder decode/separate situation of multiplexed a plurality of image/voice signals.At this moment, the video heads information in the multiplexed header can comprise the information of representing a plurality of signals.

If suitably

comprehensive execution mode

1,2,3 produces transmission code stream, can make the transmission of coded message more reliable.In the execution mode 3, as execution mode 2, can be with express time PTR (visual time reference) as mould time base and VOP incremental time.By using these information, can test as the systematicness of execution mode 2 usefulness mould time bases and VOP incremental time.

Describe the medium that storage the present invention produces information below in detail.

Figure 29 has shown to use and has wherein preserved the system that comes the reconstructed image signal from the recording medium 810 of the visual code stream of code device output of the present invention.The code stream that comprises the visual code stream of being encoded by image coding device of the present invention is stored in recording medium 810.The code stream reconstructed image signal that decoder device 820 is preserved from recording medium 810.Pictorial information output device 830 outputs one reconstructed image.Be kept at can be by the pictorial information in the recording medium of computer sense information the time as reconstruct such as personal computers, the bit stream of pictorial information is read from recording medium, and with this bit stream of image reconstruction software processes.At this moment, are displays or similar as pictorial information output device 830.The picture intelligence of reconstruct will be recorded in the storage medium (not shown) or be transferred into another device or system's (not shown) through a transmission path.

In the system of as above structure, there is the code stream of the form of above-mentioned each execution mode to be stored in recording medium 810.This code stream is characterised in that part of V OP (-image or frame) header is used as about two multiplexed informations of partial video bag (or a section, GOB etc.) head and record.Code stream reconstruct one picture intelligence of decoder device 820 from be stored in recording medium 810.Be that decoder device 820 reads code stream through a holding wire 801, and produce a reconstructed image with process shown in Figure 30.

The contents processing of decoder device 820 is described below with reference to Figure 30.

Decoder device 820 is read visual code stream from recording medium 810 one by one, and at first detects synchronous code (step S11).If detect synchronous code is VOP opening code (step S12 in be), exports last decoding VOP (frame) to pictorial information output device 830 (going on foot S13).Vop header in the decoding device 820 decoded picture sign indicating numbers behind the VOP opening code (Figure 29 " vop header ") (step S14).If be correctly decoded vop header (step S15 in be), with decoding vop header information (temporal information, VOP prediction mode information etc.).(step S16) replaces being stored in the information in the buffer storage of decoding device 820.Decoding device 820 decode then macro block data (" MB data " among Figure 29) after the vop header and decoded video bag (step S17).

If detected synchronizing signal is again synchronous mark (step S18 in be), decoder device 820 synchronous mark (RM) video packet header (macro block number (MBA), video packets quantization parameter (SQ), and header expansion code (HEC)) (step S19) afterwards of decoding again.If header expansion code HEC=" 0 " in the video packet header, (among the step S20 denying), decoding device 820 decoded video bags (step S17).If header expansion code HEC=" 1 " (among the step S20 be), and subsequently two multiplexed informations of decoding device 820 decodings (among Figure 29 " DUPH ') (going on foot S21).If can be correctly decoded multiplexed information (among the step S22 being), two multiplexed informations be compared (step S23) with the information in the buffer storage.If their unanimities (among the step S23 denying), macro block data (among Figure 29 " MB data ") after the decoding device 820 decoded video packet header and decoded video bag (step S17).If their inconsistent (among the step S23 being), decoding device 820 judges that this video packets belongs to the different VOP with last decoding VOP, and exports last decoding VOP to pictorial information output device 830 (step S24).820 two multiplexed informations with decoding of decoder device are represented the information (step S25) in the buffer storage, and next video packets of decoding (step S17).

When recording medium 810 is read visual code stream successively, repeat the above-mentioned work series that Secondary Synchronization Code among Figure 30 detects beginning, thus reconstructed video signal.

Attention can be with multiplexed voice and audio signal, data, and the code stream that the encoding code stream of control information etc. obtains is remembered and is not revised ground recording picture code stream in recording medium rather than in recording medium.At this moment, before information in recording medium was remembered in decoder device 820 decoding, demultiplexer was separated multiplexed visual code stream, the voice/audio code stream, and data and control information, decoder device 820 is separated multiplexed visual code stream then.

As shown in figure 29, remember that the information in recording medium 810 is sent to decoder device 820 through holding wire 801.Yet this information can pass the path through one and be transmitted as one line/radio frequency infrared transmission path rather than holding wire.

As above-mentioned, according to the present invention, because it is two multiplexed and write down each and remembered important information in the code stream of recording medium, even remember that the information in recording medium is wrong, or remember that the information on recording medium is wrong through holding wire or transmission path that it is sent to reconfiguration device, the energy reconstruction quality has the image of decline slightly.

Embodiments of the present invention 4 are described below.

According to the video/sound encoding device of this execution mode and overall structure such as Figure 19 of video/audio decoding apparatus, identical shown in 20.Yet the work of appropriate section but is different from execution mode 3.Below the main difference of describing.

Figure 34 A to 34C is three examples of video code flow 121A.Video encoder 111A is that cell encoding is to produce video code flow 121A with VOP (image, frame and field).Image is divided into the sub-district to be encoded that is called macro block again.

The video code flow of-VOP is since the synchronous code one VOP opening code (among Figure 34 A to 34C " VSC ") that can independently be decoded.After the VOP opening code vop header (among Figure 34 A to 34C " VH ") (also can be the image head).Vop header comprises the information of representing the VOP time, VOP coded system information, VOP quantization step information etc.After the vop header is each macroblock encoding data.

Figure 34 A is the situation that VOP is divided into the coding unit to be encoded that is called video packets (also can be section and GOB).Each video packets is made up of one or more macro block (among Figure 34 A " MB data ").When carrying out video coding, come forecasting institute to analyze the motion vector of macro block as motion vector from adjacent macroblocks with the prediction of striding a plurality of macro blocks, should be only macro block in the same video packets predict, influence other video packets to stop arbitrary transmission path mistake.

Except that first video packets, the code stream of each video packets is from the synchronous code that can independently be decoded-again synchronous mark (RM) (also can be a section opening code and a GOB opening code) and a video packet header (VPH) (also can be all head and a GOB head) among the VOP.After these yards is the data (MB data) of each macro block.Video packet header comprises the macro block number (or a slice number or a GOB number) of each macro block position in the expression video packets, video packets quantization step etc.This head also can comprise important information such as vop header information as execution mode 2.

Figure 34 B is the example of the code stream of a video coding, wherein video is divided into 2 information, promptly about the information of prediction mode and motion vector and about the residual signals in the motion compensation adaptive prediction or about residual signals being made orthogonal transform (DCT etc.) coefficient of orthogonal transform gained.In each video packets code stream, the information (among Figure 34 B " motion ') of surveying mode and motion vector about Chinese yam is arranged near reading end ' (among Figure 34 after video packet header or the vop header), be arranged on tail end about the information (among Figure 34 B " details ") of prediction residual DCT coefficient.Two category informations by motion sign (among Figure 34 B " MM ") separately.

Figure 34 C illustration is with a video coding code stream that will obtain about the coding method that the information of image sensor to be encoded is encoded with image.With reference to figure 34C, " shape " is shape information, and it is set in each video packets the information (motion) (among Figure 34 C, shape information is positioned at after video packet header or the vop header) before of surveying mode and motion vector about Chinese yam.Shape information (shape) reaches about the information (motion) of prediction mode and motion vector is separated by a shape sign (among Figure 34 C " SM ").

In the code stream of Figure 34 A to 34C, preferably synchronous code such as opening code are reached the bit position that synchronous mark is arranged on the integral multiple place of given bit number again.Figure 35 A, in the situation, the VOP opening code (VSCs) in the head of corresponding video bag reaches the integral multiple place that synchronous mark (RMs) is set at the N bit again shown in the 35B.Use this method, arbitrarily place with synchronous code and compare, the bit number that decoding device detects when detecting synchronous code is reduced to 1/N.Use this method, simplified the synchronous detecting of decoding device.In addition, the bit mode consistent with synchronous code (pseudo-synchronous code) that produce because of the transmission path mistake detected as synchronous code mistakenly, makes the probability of pseudo-synchronia can be decreased to 1/N.Even pass the wrong quality that also can improve decoded picture in path.

For in this way synchronous chicken being arranged on the precalculated position, before each synchronous code and each synchronous code, insert filling bit (among Fig. 3 SA " filling bit ') between the information.Figure 35 B is the code table of the filling bit of N=8.These filling bits be characterised in that can be anti-the direction of code stream by independent decoding, decoding device can be stipulated the length of filling bit, the widely used packing ratio top grade of being made up of " 0 " entirely of technology before not resembling.In the field 35B institute example, first of filling bit is " 0 ", and remaining is " 1 " entirely.Therefore, last position of filling bit, first bit that at first occurs when promptly reading bit before the synchronous code one by one with opposite order " 0,, can be considered to first bit of filling bit.

Because of the position of first bit in the filling bit can be described in this way, decoding device can detect the transmission error in the code stream easily.When being correctly decoded code stream, the decoding tail position of filling bit last data should be consistent with the starting position of filling bit.If the starting position of decoding tail position and filling bit is inconsistent, judge existing transmission path mistake in the code stream, this code stream can not be decoded.

When with the reverse decoded bit stream of variable length code that can reverse decoding, the starting position that decoding device must the illustration of reverse decoding.The decoding starting position of filling bit is corresponding to the last bit of filling bit.In the past in the technology, the length of the filling bit of being made up of the bit of identical value only can not be described, so decoding device can not detect the starting position of reverse decoding.In contrast, Figure 35 A, 35B allows to determine the position of first bit, determines the starting position of reverse decoding thus.

The code word of considering-comprise many " 0 " is as-synchronous code " 00000000000000001 ".As former technology filling bit only by " 0,, form, the probability that produces a bit mode consistent with synchronous code when wrong is big, therefore very likely occurs pseudo-synchronous.In contrast, use Figure 35 A, the filling bit among the 35B is because of being that " 0 ', all filling bits are " 1 ", and the Hamming distance of itself and synchronous code is big, and it is little to produce pseudo-synchronous probability except that first bit.

As above-mentioned, produce filling bit according to pre-defined rule, so decoding/separating in the multiplex machine, with filling bit in the multiplexing code stream and rule contrast, if judge the filling bit off-specification, can judge has mistake in the multiplexing code stream.Use this method, though wrong in the multiplexing code stream, thus the multiplex machine of decoding/separate can be carried out to handle and improves the quality of decoded signal to avoid the separating multiplexed/serious decline of decoded signal quality.

Except that the VOP opening code reaches synchronous mark again, motion can be indicated (MM) and shape sign (SM) be located at the integral multiple position of given bit number, and before these signs, filling bit is set as Figure 35 B.Use this, can be to shape information, prediction mode information, motion vector informations etc. are carried out error detection and reverse coding.

Figure 36 illustration multiplexer 130A.In Figure 36 example, claiming regulating course 1031A and multiplexed layer 1032A to carry out multiplexed processing on two-layer.Video code flow 121A, speech code stream 122A and data code flow 103A are input to regulating course 1031A.Be conditioned the output 1041A that layer 1031A handles, 1042A, 1043A are input to multiplexed layer 1032A.Multiplexed layer 1032A output multiplexing code stream 135A.

Figure 37 illustration regulating course 1031A handle the output code flow 1041A that video code flow 121A obtains.Processing on the regulating course 1031A is that the unit carries out with the AL-SDUs (also can be addressed location) that video code flow 121A is divided into discrete cell.Regulating course is handled the output that an AL-SDU obtains and is called-AL-PDU.Figure 37 shows the form of an AL-PDU.Added an Al head among each Al-PDU.The AL head can comprise the expression AL-PUD number and character, the information of video coding/multiplex mode etc.After the AL head AL-SOU as the AL Payload.Can establish one after the AL-SOU and be used to detect the check bit whether AL-PDU has the transmission path mistake, as the CRC check bit.

On the regulating course 1031A, processed voice code stream 122A has data code flow 103A with AL-PDUs1042A and the 1043A of output corresponding to speech code stream and data code flow in the above described manner.Yet note, the information that is provided with in each AL head, the length of CRC check bit and have/do not have etc. can be different from the AL-PDU1041A corresponding to video code flow.

The AL-PDUs1041A that on multiplexed regulating course 1031A on the multiplexed layer 1032A, produces, 1042A, 1043A.It is multiplexed with MUX-PDUs to be that the unit carries out.Figure 38 A to 38C illustration the multiplexed MUX-PDU that obtains.A multiplexed synchronous code (multiplexed sign) and a multiplexed head (MUX head) have been added among the MUX-PDU.Multiplexed head can comprise with MUX-PDU by multiplexed, the information of expression regulating course output type, and multiplex mode, and MUX-PDU length etc.

Figure 38 A is the situation that is provided with an AL-PDU among the MUX-PDU.

Figure 38 B illustration one AL-PDU be divided into a plurality of MUX-PDUs (this moment be 2).At this moment, each multiplexed head comprises that expression is divided the AL-PDU part in MUX-ODU, the information of the sequence number in whole AL-PDU, or the AL-PDU that is divided of expression partly is among the Al-POU first or the information of last part.

Figure 38 C illustration one MUX-PDU a plurality of AL-PDUs are arranged.In Figure 38 C example, corresponding to the AL-PDU (video AL-PDU) of video code flow and corresponding to the AL-PDU (audio A L-PDU) of speech code stream by multiplexed.At this moment, multiplexed head can comprise among the expression MUX-PDU information on border between a plurality of AL-PDUs.Substitute, can between AL-PDUs, establish the sign on an expression border by boundary.

As above-mentioned, be the cell processing code stream with AL-SDUs or addressed location on the regulating course.How Figure 39 A to 39C illustration one video code flow is divided on regulating course.

Each shows the situation that a VOP is set as an addressed location Figure 39 A to 39C.Figure 39 A to 39C is respectively corresponding to the video code flow among Figure 34 A to 34C.

Each shows the situation that a video packets is set as an addressed location Figure 40 A to 40C.Figure 40 A to 40C is respectively corresponding to the video code flow among Figure 34 A to 34C.

As Figure 34 B, shown in the 34C, video packets is divided into shape information, motion vector information, and the DCT coefficient information is when encoding it, addressed location also can be correspondingly by segmentation.Figure 41 A, 41B are respectively corresponding to the video code flow among Figure 34 B and the 34C.The shape sign (SM) and the motion sign (MM) on expression border are arranged on shape information (shape), about the information (motion) of prediction mode and motion vector, and about between the information (details) of an error signal and a DCT coefficient with the generation addressed location.

During multiplexed synchronous code,, can divide the starting position of corresponding addressed location from these yards and logo area as the above-mentioned AL boundary marker that in the code stream of multiplexed layer, adds border between expression MUX-PDUs or AL-PDUs etc.At this moment, the synchronous code in the addressed location head can be removed from video code flow.Each is the situation that a VOP is set up as an addressed location for Figure 42 A to 42C.At this moment, the VOP opening code in the removable vop header, each is the situation that a video packets is set up as an addressed location for Figure 43 A to 43C.VOP opening code in removable each video packets and synchronous mark at this moment again.Each illustration of Figure 44 A to 44B shape information (shape), each forms an addressed location about the information (motion) of prediction mode and motion vector and about the information (details) of residual signals and DCT coefficient.At this moment, can remove VOP opening code and synchronous mark and expression " shape " again in each video packet header, " motion ", the motion sign (MM) on the border between " details " and shape sign (SM).

As Figure 45 A, shown in the 45B, in the addressed location one or more video packets can be arranged.At this moment, shown in Figure 45 B, only can remove VOP opening code or synchronous mark again in each addressed location head.Similarly, Figure 34 B, in the code stream of 34C, each addressed location can be composed of a plurality of video packs.

As Figure 34 B, shown in the 34C, video packets is segmented into " shape ", " motion ' and " details " when encoding it, each addressed location can be by one group of shape data of a plurality of video packets, and exercise data or detail data are formed.Each illustration of Figure 46 A to 46D code stream among Figure 34 B is carried out such processing, and each addressed location is made up of one group of exercise data or detail data.Establish a vop header and video packet header before in " motion " of each video packets.

Addressed location can by many groups with VOPs or arbitrarily a few days video packets be that " motion " of unit reaches " details " and form.

In this addressed location form, can video packets " motion ' establish a synchronous code with the boundary of " details ".Figure 46 B illustration the boundary between exercise data establish a synchronous code (RM).Figure 46 C and each illustration of 46D on the border between exercise data and the boundary between detail data set up synchronous code (RM).Figure 46 D illustration each addressed location the head in establish a synchronous code (VSC)." motion ' and " details ' available different sign indicating number.As move the sign be used for " motion ", synchronous mark is used for " details " again.

In the video code flow, each addressed location also can be by one group of shape data shown in Figure 34 C, and exercise data or detail data constitute.

As above-mentioned, can be from the different code stream of significance level, as " shape "; " motion ', and " details " collected the identical code stream of significance level and adopted and form each addressed location; and provide in various degree error protection (as using error correction and error detection code, and passing again) for each addressed location.Use the method, can come the execution error protection according to the significance level of corresponding code stream, when transmission path is wrong, to improve the quality of decoded picture.Generally, shape information (shape), or when in mode information or the motion vector information (motion) error of transmission being arranged, the decoded picture quality reduces greatly.For avoiding it, can use the error correcting code that is suitable for shape and motion so that strong error protection to be provided.In contrast, if in the residual signals (details) the transmission path mistake is arranged, image quality descends little.Its error protection need not be too strong, therefore can reduce error correcting code, the redundancy that error detection code etc. produce.

Above-mentionedly remove the situation of synchronous code from video code flow, multiplexer 130A can remove the synchronous code among the video code flow 121A, or video encoder 111A removes the synchronous code that will be sent in the video code flow of multiplexer in advance.

In the arbitrary example of Figure 39 A to 460, the length of each addressed location can be set as the integral multiple of predetermined length (as byte).As Figure 35 A, shown in the 35B, one video code flow is that the unit is divided into video packets or VOPs with the N bit, and set filling bit before synchronous mark or each opening code again at each, if each addressed location comprises these filling bits, the length of each addressed location can be established the integral multiple of salty predetermined length (as byte).

Handle if video code flow is not carried out this, can add filling bit is made as predetermined length (as byte) with the length with each addressed location integral multiple at the tail of each addressed location.Can use shown in Figure 35 B as filling bit.At this moment, can be as in video code flow, inserting filling bit, the use filling bit detects the mistake in the code stream.In addition, except that video code flow, also can add filling bit in speech code stream and the data flow, be made as the integral multiple of predetermined length (as byte) with length each addressed location.

On the multiplexed layer, when multiplexed Payload comprised the bit mode identical with multiplexed synchronizing signal, decoder error ground judged that this pattern is multiplexed synchronous code.Detected to erroneous results the border between MUX-PDUs.Promptly reproduce pseudo-(also can be truth and false) synchronously.If video encoder produces a video code flow, wherein in the place that is not each synchronous code, do not produce with video code flow in the consistent bit mode of synchronous code (as VOP opening code or synchronous mark again), can use an audio video synchronization sign indicating number to detect whether to have in the multiplexed layer synchronously pseudo-.

The head of MUX-PDU is adjusted to the head of an AL-PDU and sentences formation one MUX-PDU.Each shows a form Figure 38 A to 38C.Added an audio video synchronization sign indicating number in the head of each AL-SDU (addressed location).With this, through a multiplexed head or an AL head, a multiplexed synchronous code and an audio video synchronization sign indicating number adjoin each other.If decoding device detects a multiplexed synchronous code mistakenly, this device then detects the multiplexed head of contiguous this detected sign indicating number, an AL head, and an audio video synchronization sign indicating number.Yet because tested sign indicating number is a pseudo-synchronous code, as a multiplexed head, an Al head and an audio video synchronization sign indicating number are decoded as if tested information, obtain wrong information.So the multiplexed head that the demultiplexer check is decoded, the AL head, and whether the audio video synchronization sign indicating number is correct information.If judge that they are incorrect, then tested multiplexed synchronous code is a pseudo-synchronous code.

Figure 48 is second example of multiplexer.In this example, multiplexer is divided into two-layer, i.e. a FlexMUX layer and a TransMux layer.The FlexMux layer is divided into an adjusting sublayer (AL) and and is multiplexed in layer.The TransMux layer is divided into a protection sublayer and a TransMUX sublayer.

Figure 49 is the code stream that produces on the

F1exMUX layer.Label

1061A and 1062A refer to regulate the code stream that produces on sublayer and the Mux sublayer respectively.Comprise the information of representing to treat the multiplexed information type, AL head (head) 1064A of the information of express time etc. is input to and regulates the sublayer.In addition, multiplexedly treat the multiplexed video that comprises, the Payload 1066A (Payload) of audio frequency and data code flow is to produce an AL-PU.On the Mux sublayer, add expression Al-, information (length) 1069A of the index 1068A of PDU type and channel number and expression AL-PDU length is to produce a FlexMux-PDU.

The FLexMux-PDU that produces on the FlexMux layer is input to the TransMUX layer.The structure of multiplexer among TransMUX layer cun employing Figure 36, at this moment, the protection sublayer is corresponding to the regulating course 1031A among Figure 36, and the transmax sublayer is corresponding to the multiplexed layer 1032A among Figure 36.Substitute, structure can be used for the transmax sublayer among Figure 36, and need not protect the sublayer.

The structure of noting multiplexing code stream in the execution mode one to three can adopt that multiplexed wherein each has length to be set as the element number flow structure of predetermined length integral multiple by filling bit, and the synchronization code word of collecting no less important is to form the structure of addressed location.

As Figure 45 A, 45B is the same, and when a plurality of video packets were set as an addressed location, mode that can Figure 50 was arranged addressed location and the border between synchronous mark again in a frame.With reference to Figure 50, each hollow plate refers to have again and again the macro block (being first macro block of each video packets) of synchronous mark, and each full hawk refers to the position of each macro block in each addressed location.In this image, because of the human body image overweights background information, preferred human figure's image information has high anti-transmission path mistake.Arrange video packets so put many synchronous marks more at interval with short, thereby strengthen anti-mistake, can make image from the transmission path mistake, recover fast like this in the human body image area.In contrast, because background parts is not too important, can arrange less synchronous mark again in the background parts to increase the interval of its video packets.

When encoding a frame with raster scan order from left upper end macro block to bottom righthand side macro block, the mistake in the given macro block will reach the macro block of bottom righthand side.When especially mistake reached an important area, image quality reduced greatly, so the macro block when important area begins is configured to first macro block in the addressed location, to avoid the erroneous effects important area in another macro block.In Figure 50 example, be first macro block in the addressed location as the macro block of the human body of important area image left end.

If the error protection intensity of an addressed location can change, can change intensity according to the importance in zone in the frame.With reference to Figure 51, it is assigned to the human body parts as important information a district (high QoS) of the strong error protection of light gray (shade) Qu Zhiyou.Each illustration of Figure 52 A to 52C corresponding to the form of the addressed location of this image.With reference to figure 52A to 52C, light gray (shade) is distinguished corresponding to light gray macro block among Figure 51, and it has strong error protection.

One video packets is divided into " fortune merit " and reaches " details " and when coding, and shown in Figure 52 A to 52C, " motion " is located at the first half of an addressed location, and " details " is located at its latter half.In addition, the important district of light gray indication can be located at respectively that first half and latter half substitute among Figure 51, " motion ' and " details ", can be located at different addressed locations.And stronger error protection is provided for the first half of corresponding addressed location.Use this structure, can give in the code stream in important district and provide stronger error protection than other parts prior " motion " part.

As above-mentioned, according to the present invention, by arranging the form of synchronous mark and use addressed location, extra head (redundancy) that can be less is realized high anti-mistake again.Generally, when having used synchronous mark again and strong poor protection being provided, extra head has increased.Yet; if distribute to as many synchronous marks again such as human body of important information to strengthen error protection to it; and give little important zone such as background parts synchronous mark seldom to weaken error protection; evenly given whole image with synchronous mark again; the also uniform situation of its error protection is compared; under same average amount the outside, important information has high anti-mistake.

When having distributed many synchronous marks again for important information among Figure 51 such as human body parts, correspondingly the length of each video packets significantly reduces.Therefore distribute to an addressed location as if each video packets, then based on the AL head, multiplexed head, the extra head of multiplexed synchronous code etc. increases greatly.At this moment, as Figure 45 A, shown in the 45B, preferably be provided with a plurality of video packets in the addressed location to reduce extra head.

Figure 51 shows the circuit structure of the encoder 103 of code device among Fig. 1.According to this structure, the pictorial information that inputs to encoder 103 at first is sent to movement compensating circuit 1036.At this moment, movement compensating circuit 1036 is at input image information and be stored between the former frame information in the frame memory 1035 and carry out motion compensation, and subtracter 1030 calculates the information of compensation back present frames and the difference of former frame information.The different information discrete cosine conversion circuit of contrast (DCT) 1031 carries out dct transform, and is quantized device 1022 quantifications, and object information is sent to variable length encoder 1038.

The information that quantizer 1032 comes out is by de-quantizer (IQ) 1033 de-quantizations.The result is carried out dct transform by back-discrete cosine inverting circuit (IDCT) 1034.The output of back-discrete cosine inverting circuit 1034 is added in adder 1037 with the motion compensation information that movement compensating circuit 1036 comes out; The information that adder 1037 is come out becomes the decoded picture information of present frame, i.e. local decode image.This local decode pictorial information is stored in frame memory 1035 with the moving compensating data as the coding next frame.

Send quantitative information in variable length encoder 1038 by variable-length encoding.The result is sent to important information specified circuit 1039.This important information specified circuit 1039 only extracts the chapter of outside appointment and wants message part from variable-length encoding information, and the high-ranking military officer delivers to important header reconfigurable circuit 104.Notice that all information are sent to Bit String reconfigurable circuit 107, no matter whether important information specified circuit 1039 has specified important information.

Figure 54 is the circuit structure of decoder 124 in the decoding device shown in Figure 2.According to this structure, the information that demultiplexer 121 is sent here is temporary in reception buffer 1241.Thus, the decoding start position information that synchronizing indicator 122 is sent here responds, and subsequently header is sent to a header decode device 1242.Decoder 1242 decoding headers.At this moment, the header of the superiors of at first decoding, whether decoded information is sent in the checking information vicious error detection circuit 125.Header according to the appointment of appointed information decision circuit 123 decoding subsequent layer.Promptly specify when important header is arranged, the information in decoding precalculated position is as important header.This important information is sent to important information circuit 126 to be temporary in wherein.Important information circuit 126 detects the superiors' header when wrong, because the important header of important information circuit 126 is sent back to decoder 1242, by using the important header of important information circuit 126, the decoding processing that decoder 1242 continues subsequently.

Information (actual image information) after the de-quantizer 1243 de-quantization headers.Its result is sent to idct circuit 1244.Idct circuit 1244 calculates the inverse discrete cosine transform of de-quantization information, and difference signal between decoding present frame and the former frame.Adder 1245 with this decoded differences signal add to the former frame information that is stored in the frame memory 1246 (carrying out former frame information after the motion compensation) with motion vector information.The reconstructed image information that this addition results signal is exactly a present frame.This information is sent to D/A circuit 127, and is stored in frame memory 1246.

Coding/decoding device structure and flow structure in the above-mentioned execution mode can mix mutually, if desired.In addition, respective coding/decoding work can be replaced by the process based on software control.Corresponding software programs and encoding code stream can be stored in the storage medium.

As above-mentioned, according to the present invention, because code flow structure itself has anti-mistake, even important information such as header are wrong, this information can not be used for decoding, can correctly continue decoding as an alternative by the fresh information that uses the appointed information appointment.In addition; owing to provide strong error protection with error correcting code and error detection code for the image head in important information such as the video chicken stream and section head; with do not provide the conventional video code device of enough error protections and traditional multiplex machine to compare to important information; even the transmission path mistake is arranged, also can decode a high quality video signal.In addition, be the multiplexed unit code stream of predetermined number integral multiple owing to produced length by the adding filling bit, can detect mistake easily.Also have,, can provide error protection according to the significance level of code word by for having formed addressed location by the code word of collecting no less important.The result has to stagger the time at transmission path and has improved the quality of decoded picture.

Claims

1. a transmission comprises the following steps: to its information transferring method that has added the coded message of the required header of coded message of decoding

Add the reconfiguration information of the content that allows reconstruct header or division header information to data flow; And

Transmission has been added the data flow of reconfiguration information to it, thereby might be by substituting header or division header information with reconfiguration information at the receiver end decoded data stream.

2. according to the information transferring method of claim 1, also comprise:

Further add and show the appointed information of having added reconfiguration information to data flow to header.

3. according to the information transferring method of claim 1, wherein said coded message comprises the video code flow that obtains by the compressed encoded video signal, and described reconfiguration information comprises the information of the demonstration time of each frame of video of representing video code flow.

4. a transmission comprises the information transferring method of the coded message of the video code flow that obtains by the compressed encoded video signal, and wherein that decoding and coding information is required header adds coded message to, and this method comprises:

Add to allow the reconfiguration information of the content of reconstruct header or division header information, described reconfiguration information to comprise the information of predictive mode of each frame of video of expression video codes string to data flow; And

5. according to the information transferring method of claim 4, also comprise:

6. according to the information transferring method of claim 4, wherein said reconfiguration information comprises the information of the demonstration time of each frame of video of representing video code flow.