JP2004357321A - Encoding apparatus in information transmission system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To normally attain decoding processing even on the occurrence of an error in important information such as header information by providing error robustness for a bit stream itself in an information transmission system for transmitting an image or the like subjected to compression encoding. <P>SOLUTION: A bit stream reconstruction circuit 107 adds information having already been transmitted from an upper layer or part of the information, information having already been transmitted in the same layer or part of the information, or information for restoring contents of the information having already been transmitted in the upper layer or the same layer or contents of the part information to encoded information as restoration information including information denoting a prediction mode of each image frame of an image encoding stream in order to restore the encoded information including the image encoding stream obtained at an encoding circuit 103 to apply compression encoding to the image signal and thereafter uses a designation information insertion circuit 106 to insert designation information with a prescribed bit pattern denoting the addition of the restoration information into the header information. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an encoding device in an information transmission system for transmitting an encoded moving image / still image using a wired communication network such as ISDN or the Internet, or a wireless communication network such as PHS or satellite communication.

  In recent years, with the development of digital coding technology for images and other types of information and broadband network technology, applications using these technologies have been actively developed, and compressed and coded images are transmitted using communication networks. A system has been developed.

  For example, in videophones, videoconferencing systems, and digital television broadcasting, video and audio are each compression-encoded to a small amount of information, and the compressed video and audio code sequences and other data code sequences are multiplexed. A technique of transmitting / accumulating a single code string collectively is used.

  Techniques such as motion compensation, discrete cosine transform (DCT), sub-band coding, pyramid coding, and variable-length coding, and methods combining these techniques have been developed as compression coding techniques for moving image signals. International standards for video coding include ISO MPEG1, MPEG2, and ITU-T H.264. 261, H .; 262, H .; H.263, and an international standard system for multiplexing a moving image, a code string obtained by compressing an audio / audio signal, and other data, include the ISO MPEG system and ITU-TH. 221, H.R. 223 are present.

  In a conventional moving picture coding method such as the above-mentioned moving picture coding international standard method, a moving image signal is divided into frames, and the frames are further divided into small areas, and each unit such as GOB, macro block, or the like is divided. Encoding is performed, and header information indicating an encoding mode and the like is added to each frame, GOB, and macroblock. These header information are necessary information for decoding the entire frame, GOB, and the like. For this reason, if an error due to the transmission path / storage medium is mixed in the header information and the video information cannot be decoded correctly by the video decoding device, the entire frame including the header information, GOB, etc. cannot be correctly decoded, and the video decoding device does not. In this case, the quality of the reproduced moving image will be greatly deteriorated.

  That is, when transmitting an image that has been compression-encoded using a communication network, the receiving side needs to perform a decoding process of reproducing meaningful information from the transmitted 0/1 bit string. To this end, the above-mentioned header information is very important as information indicating under what rule a certain unit of a bit string has been encoded. The header information includes, for example, the prediction type of the currently decoded frame (whether it is intra-frame encoding or inter-frame encoding, etc.), and information (time, Reference), or step size information when performing quantization. If the header information is lost, information transmitted thereafter cannot be correctly decoded.

  For example, suppose that although the prediction type of the frame originally indicates that the coding is between frames, an error is mixed in the bit string for some reason and the bit pattern changes to a bit pattern indicating the inside of the frame. In this case, even if the subsequent actual information is transmitted correctly, the decoding side determines that the signal is the result of the intra-frame encoding, so that it will not be correctly decoded eventually. Therefore, the quality of the reproduced moving image in the moving image decoding apparatus is greatly deteriorated.

  Such mixing of errors frequently occurs particularly when a system that transmits / stores a moving image via a wireless transmission path or the like, such as a wireless videophone, a portable information terminal, or a wireless digital television receiver, is used. Conventional image transmission uses a system using a wired communication network, and satellite communication with a very low error rate is assumed even when a wireless communication network is used. Therefore, sufficient consideration has not been given to the error resistance of the structure of the coded sequence to be transmitted itself, and the transmission path error protection for important information such as header information has not been sufficient.

  In PHS, which will become the mainstream of mobile communication in the future, the error rate will be about 100,000 to 1,000,000 times that of satellite communication. Therefore, it is not enough to perform error correction on the encoded bit sequence as in the past. It becomes impossible to correct. Also, in the Internet, which is expected to become the mainstream of communication in the future like PHS, it is not clear statistically how and when errors will be mixed, and there may be cases where appropriate error correction cannot be performed. . In addition, in the case of the PHS or the Internet, some information in the code string may be lost, and a situation that cannot be dealt with theoretically by error correction may occur. Therefore, it is necessary to give the structure of the code string itself an error resilience capability.

  As described above, conventionally, sufficient consideration has not been given to the error resistance of the structure of the coded sequence to be transmitted itself, and in particular, important information such as header information that induces large image quality degradation when a transmission path error enters. Did not consider transmission line errors sufficiently.

  The present invention has been made in view of such a point, and by giving an error resilience capability to the structure of a coded sequence, an error occurs in important information such as header information due to a transmission line error. It is another object of the present invention to provide an encoding device in an information transmission system capable of decoding an image signal with high quality.

  The basic concept of the present invention is to realize a system configuration in which important information such as header information can be repeatedly transmitted at an arbitrary timing according to an instruction of a certain instruction information. Is to have an error resilience capability.

  For this purpose, the present invention divides encoded information including an image code sequence obtained by compression-encoding an image signal into two or more layers, and assigns a synchronization signal to each layer and a signal necessary for decoding. In the information transmission system for transmitting by adding the header information, the upper layer is used as restoring information including information indicating a prediction mode of each image frame of the image code sequence for restoring the encoded information. Information already transmitted in or part of the information, or information already transmitted in the same layer or part of the information, or information already transmitted in the upper layer or the same layer or part of the information By providing an encoding device having means capable of transmitting information capable of restoring the content of information, the structure of the code string itself has an error resilience capability. Myself understood, it is made possible to perform the decoding processing properly even error occurs in the header information to thereby.

  As described above, according to the present invention, by giving error resilience capability to the structure of a coded sequence itself, it is assumed that important information such as header information contains errors and cannot be used for decoding. Also, the decoding process can be correctly continued by using the new information indicated by the instruction information.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a configuration of an encoding device according to an embodiment of the present invention. A moving image captured by the camera 101 is converted into a digital signal by an A / D converter 102 and input to an encoding circuit 103. In the encoding circuit 103, high-efficiency compression encoding of a moving image signal is performed by DCT transform, quantization, variable length encoding, inverse quantization, inverse DCT transform, motion assurance, etc., and the encoded data sequence is converted. Generated. The important header information in the encoded data sequence is input to the important header information reconstructing circuit 104, where it is temporarily stored. After the encoding circuit 103, there is a bit string reconstructing circuit 107, where a final code string sent to the transmission path, that is, a data stream conforming to MPEG2 or the like is determined.

  In the bit string reconstruction circuit 107, first, a synchronization signal determined by the synchronization signal circuit 105 is added to the beginning of the bit string in a certain fixed bit string, and thereafter, instruction information is inserted into the bit string by the instruction information insertion circuit 106. Is done. This instruction information indicates the addition of important header information. By inserting this instruction information into a bit string, it becomes possible to add important header information to the bit string. Next, necessary important header information is extracted from the important information reconstructing circuit 104 and added to the bit string. The details of the bit string configuration here will be described later with reference to FIGS.

  The bit sequence finally determined by the bit sequence reconstruction circuit 107 is multiplexed with other encoded audio information, character information, and the like by the multiplexing circuit 108 and transmitted to the transmission line 110. Note that the user can freely specify what is included in the header information as important header information from outside the encoding circuit 103.

  FIG. 2 shows the configuration of the decoding unit according to the present invention.

  The transmitted bit string is separated by a separation circuit 121 into image information, audio information, character information, and the like. First, the synchronization detection circuit 122 detects the synchronization of the bit sequence of the image information, thereby detecting the decoding start position in the bit sequence. The information is sent to the decoding circuit 124, from which the decoding process is started or restarted. First, decoding is started from the header information of the uppermost layer. These decoded signals are checked by the error check circuit 125 as to whether or not an error is mixed. If it is confirmed that an error has occurred, that part cannot be used, and the fact is notified to the important information circuit 126. When decoding the header information of the next layer, after detecting synchronization in the synchronization detection circuit 122, the same bit string is transferred to the instruction information determination circuit 123, and the content of the instruction information is examined here. As a result, it is detected whether or not important header information has been added, and if added, the type and addition position of the important header information are detected. Based on this detection result, an operation instruction is given from the instruction information determination circuit 123 to the decoding circuit 124. The decoding circuit 124 decodes the header information of the current layer and the important header information added thereto. The decoding result of the important header information is transferred to the important information holding circuit 126, where it is once held. If the error check signal is received from the error check circuit 125, it is known that the important header information in the upper layer cannot be used. Therefore, the decoding circuit 124 checks the important header information transmitted in the current layer. By substituting, the decoding process for the code string subsequent to the current layer is continued. The information such as the decoded image is returned to the analog signal in the D / A circuit 127 and displayed on the monitor 128.

  Next, the structure of the image code string used in the present embodiment will be described.

  FIG. 3 is a conceptual diagram when an image is divided into a plurality of layers.

  One frame 200 is divided into a number of slices (macroblock lines) 201 in which macroblocks of 16 pixels × 16 pixels are collected (FIG. 3A). On the other hand, each slice 201 is constructed as a set of several macroblocks 203 (FIG. 3B). The case where the entire screen is captured as the frame 200 is the highest layer, the case where each frame 201 is captured is the next layer, and the level 203 is the next layer.

  FIG. 4 is a diagram showing an example of the data structure of each layer shown in FIG.

  FIG. 4A shows a bit sequence of the uppermost frame layer corresponding to FIG. 3A, and FIG. 4B shows a conventional example of a bit sequence of a slice layer corresponding to the slice of FIG. 3B. . On the other hand, FIG. 4C shows an example of a new bit sequence proposed here for a slice layer corresponding to the slice of FIG. 3B.

  As shown in FIG. 4A, a frame layer, that is, an image code string of one frame starts with a synchronization signal (picture start code; PSC) indicating a picture start position. The PSC is followed by a time reference (TR) indicating the timing of reproducing the frame, type information (PT) of predictive coding such as intra-frame / inter-frame, and then quantization step size information (PQ). These TRs, PTs, and PQs are information necessary for all the decoding processing or display of the screen. If these information are destroyed due to an error or the like, even if the synchronization is re-established in a subsequent layer, the decoding or display is performed. Will not be done correctly. The lower layer information is stored in Data after the PQ, and a typical bit string is shown in FIG.

  As shown in FIG. 4B, in the slice layer, the image code string of each slice 201 starts with a synchronization signal (SSC) indicating the start, then predictive type information (SPT), and then. The slice number (SN) and finally the quantization step size information (SQ). Subsequent Data is information of a lower macroblock layer.

  Next, the structure of the slice layer used in the first embodiment will be described with reference to FIG.

  As described above, the information in FIG. 4A is important information, and if this part cannot be used, the screen cannot be correctly decoded even if the information of the slice layer below it is not destroyed. . Even if the information of FIG. 4A cannot be used, in order to correctly decode the information of the lower slice layer, it is necessary that the contents of the header information of FIG. . Therefore, in the first embodiment, a code of a predetermined bit pattern indicating the instruction information is prepared in the SPT, and when the code appears, the header information transmitted in FIG. To be transmitted. In this example, the TR and the PT are transmitted (in this case, the SPT is used as the instruction information, and the PT is necessary because the SPT does not represent the original prediction type). If no error is mixed in the frame layer of FIG. 4A, this information (TR, PT) is not used. However, if the information of the frame layer is destroyed by an error or the like, the information shown in FIG. The decoding process can be continued by substituting the information in 4 (c).

  FIG. 5 is another example replacing FIG.

  The frame layer shown in FIG. 5A is the same as that shown in FIG. 4A, but the instruction information inserted into the header information of the slice layers shown in FIGS. ing. Although instruction information is prepared in the SPT in FIG. 4, a new bit (IS) is inserted here. If the IS indicates that the important information of the frame layer follows, the TR is transmitted next to the IS as shown in FIG. In the case of being destroyed by, for example, the TR of the slice layer is used. In this case, since the SPT indicates only the prediction type, it is not necessary to transmit the PT again in the slice layer as in the case of FIG.

  FIG. 5D is a modification example of FIG. 5B, and is an example in which the SPT is not transmitted in the slice layer. At this time, when the important information of the frame layer is retransmitted according to the instruction of the IS, the TR and the PT are required in the slice layer as shown in FIG.

  FIG. 6 is a conceptual diagram in the case where the screen is composed of a single layer, and an example of a bit string at that time.

  In this case, the screen is simply divided into blocks (macro blocks) as shown in FIG. In this case, as shown in FIG. 6B, one frame of the image code string has a structure in which the entire screen is synchronized with only one synchronization signal PSC. At this time, since the TR and the PT are still important, if these pieces of information are destroyed, decoding cannot be performed even if the subsequent information is correctly transmitted. Therefore, a mechanism for transmitting such important information again by some method is effective. In particular, when a random error occurs, the probability of destruction of either information is significantly reduced as compared with the case where information is transmitted only once. Also, when a burst error occurs, the probability that both of these retransmission information are destroyed can be reduced by keeping the retransmission information apart from the first information to some extent. In the example of FIG. 6B, the IS is inserted after the important information TR, PT, PQ, etc., and the TR, PT, etc. can be inserted after that according to the instruction of this signal. For the above-mentioned reason, it is effective to keep the position of the instruction information IS away from the important information for, for example, more than the statistical duration of the burst error.

  FIG. 7 shows another configuration example of the encoding unit according to the present invention.

  The image captured by the camera 301 is converted into a digital signal by an A / D converter 302 and input to an encoding circuit 303. After the encoding circuit 303, there is a bit string reconstructing circuit 307, where the final bit string sent to the transmission path is determined. In particular, when using a network in which an error is likely to occur, it is usual to perform a refresh operation without prediction at regular intervals in order to minimize the propagation of errors that cannot be completely recovered. Although this refresh can put the entire surface in that mode (in this case, the prediction type of the frame becomes the intra-frame), the refresh (intra-frame encoding) generates the generated information as compared with the inter-frame encoding. The amount is so large that it is particularly difficult to use it for low bit rate coded transmission. Therefore, it is preferable to use a method in which refresh is performed on only a part of one frame and refresh of a position corresponding to the entire frame is completed over a certain period of time. If an error is detected on the decoding side, it is also important to issue a retransmission request and have only that part retransmitted.

  In order to realize these processes, it is necessary that prediction switching within a frame or between frames can be performed during encoding by the encoding circuit 303. When refreshing only a certain portion (here, for example, the specific slice shown in FIG. 3 as an example) is performed, the prediction type of this slice is different from the prediction type of the previous slice. Information. Also, in the case of refresh, the quantization step size is significantly different from that in the case of inter-frame prediction, so this information is also important.

  In the encoding device of FIG. 7, when encoding for refresh is performed by the encoding circuit 303, the information is sent to the instruction information insertion circuit 305. On the other hand, the important information for the refresh is stored in the encoding process change information circuit 306 in advance. In the bit string reconstruction circuit 307, the synchronization signal determined by the synchronization signal circuit 304 is added to the head of the bit string of the slice that has been coded for refreshing, and thereafter, the refresh is performed by the instruction information insertion circuit 305. Is inserted. In this state, since the above-mentioned important information necessary for decoding the refreshed image can be added, necessary important information is extracted from the encoding process change information circuit 306 and added to the bit string of the slice. The details of the bit string configuration example here will be described with reference to FIG.

  The bit string finally determined by the bit string reconstructing circuit 307 is multiplexed with other encoded voice information, character information, and the like in the multiplexing circuit 308 and transmitted to the transmission path 310. At the time of refreshing, what kind of information is added as important information can be freely specified by the user from outside the encoding process change information circuit 306.

  FIG. 8 is a configuration example of a decoding device corresponding to the encoding unit in FIG. The transmitted code string is separated by a separation circuit 320 into image information, audio information, character information, and the like. First, the synchronization detection circuit 321 detects the synchronization of the bit sequence of the image information, whereby the decoding start position in the bit sequence is detected. The information is sent to the decoding circuit 323, from which the decoding process is started or restarted. Further, the bit string is also transferred to the instruction information determination circuit 322, where the content of the instruction information is determined. In the case of a configuration capable of realizing refresh, it is sufficient that the type of decoding processing can be changed in accordance with the intra-frame / inter-frame predictive coding type according to this instruction information, so the intra-frame decoder 324 in the decoding circuit 323 and the inter-frame decoding The switch for switching the unit 325 is switched by a signal from the instruction information determination circuit 322. Then, decoding processing is performed by the intra-frame decoder 324 on the slice for refreshing. This intra-frame decoding process is controlled according to important information such as the above-described quantization step size. The image information decoded by the intra-frame decoder 324 or the inter-frame decoder 325 of the decoding circuit 323 is converted back to an analog signal by the D / A circuit 326 and displayed on the monitor 327.

  FIG. 9 is a diagram showing a state in a frame when refreshing is performed and a structure of a corresponding image code string.

  The frame 351 is divided into a plurality of slices. Here, for example, a case is considered in which a slice 353 for refreshing is transmitted after a slice 352 for performing inter-frame coding (FIG. 9A). The next slice 354 is again inter-frame coded. FIG. 9B shows the state of the bit string transmitted at this time. Each part 361, 362, and 363 in this bit sequence corresponds to slices 352, 353, and 354 in FIG. 9A, respectively. In the SPT2 included in the bit string of the slice for refreshing, indication information indicating that the refresh is performed using intra-frame coding is inserted. Subsequent SQ2 indicates a quantization step size prepared for refresh. All of Data2 is decoded as a result of intra-frame encoding.

  FIG. 10 is a diagram showing another example of the contents of important information, in which FIG. 10A shows a frame and FIG. 10B shows a bit string of a slice layer.

  In the slice layer, the information that comes after the SPT serving as the instruction information is TR in FIG. This may represent the display timing as it is, but in some cases, the number of bits involved in the representation may be large. In order to avoid this, in FIG. 10, a method of encoding the difference between the information and the previously transmitted information corresponding to the information, which is a process generally performed in the compression encoding, is adopted.

  That is, when 0 to 255 are considered as TR, 8 bits are required to express these as they are. However, considering this, for example, in the case where the constraint condition that there is no frame dropping of three or more frames is satisfied, at this time, frames that are adjacent to each other at the time of display will not be separated by at least three frames. It is sufficient that four states (the number of dropped frames is 0, 1, 2, 3) can be expressed as a typical time reference. At this time, it is sufficient that 2 bits are used as TR, and the number of bits can be reduced. However, in this case, since the information immediately before decoding is required, the display timing is not determined only by this TR.

  FIG. 10B is an example of a bit string when the above-described difference TR (DTR in the figure) is transmitted as important header information in the slice layer. After the DTR is decoded, the true TR of the current frame can be calculated by adding the DTR to the TR information of the pit string of the frame layer corresponding to FIG. 10A of the previous frame that has already been decoded. .

  FIG. 11 is a diagram showing a circuit for performing the decoding process in the case as shown in FIG. This figure operates by replacing the decoding part of FIG. First, the decoding start position of the bit string sent from the separation circuit 121 is known by the synchronization detection circuit 122, and the TR, PT, etc. of the frame layer are decoded by the decoder 401. At the same time, the decoded TR information is stored in the memory 2 (404). The header information is checked for errors by the error check circuit 402, and the result is transferred to the memory 1 (403) (the processing of FIG. 10A). On the other hand, for the slice layer in FIG. 10B, the SPT is first determined by the instruction information determination circuit 123, and the subsequent DTR and PT are decoded at 401. The DTR is transferred to 403. If it is found from the information from 402 that the header information (TR) of the upper frame layer cannot be used due to an error or the like, a request signal is sent from 403 to 404, and the previous frame already stored here is output. Is transferred to 403. In 403, the TR and the DTR in the current frame are added to create a TR of the current frame, and the TR is returned to 401 to continue the decoding process. It should be available for similar processing in the next frame.

  FIG. 12 shows an example in which an unused pattern among bit patterns prepared for other header information is used as instruction information.

  Here, it is assumed that a 2-bit bit pattern is assigned in advance as the SPT. There are three types of prediction: I (intra-frame coding), P (forward prediction coding), and B (bidirectional prediction coding), and bit patterns of 00, 01, and 10 are assigned, respectively. Therefore, since the information for “11” is not used, this code is used as the instruction information. That is, when the SPT is “11”, it is not a prediction type, but indicates that important information follows. The important information indicated by the instruction information may be header information (TR, PT, PQ) or a part thereof, or may include subsequent data (for example, Data in FIG. 4A). These can be changed according to the requirements of the system, the frequency of occurrence of errors in the network, the required coding rate, and the like.

  As described above, in the first embodiment, even when important information such as header information is lost, information for restoring the important information is additionally transmitted according to an instruction of certain instruction information. Even if an error or the like occurs and cannot be decoded, the decoding process can be correctly continued using the instruction information transferred thereafter and the restoration information specified by the instruction information.

  Next, a second embodiment of the present invention will be described.

  The overall configuration of the encoding device according to the second embodiment is the same as the encoding device in FIG. In the second embodiment, the image signal is encoded by dividing each frame (also referred to as a picture or VOP) into small areas (dotted areas in the figure) called macroblocks as shown in FIG. Further, when an error is mixed in an image code sequence obtained by encoding an image signal, a frame is divided into a video packet composed of one or a plurality of macroblocks (a solid line area in FIG. ) Is encoded.

  FIG. 14 shows an example of an image code string output from the encoding device. FIG. 14A shows an image code string of one entire frame (VOP). At the head of the VOP, a VOP start code (VSC in the figure), which is a synchronous code that can be uniquely decoded, and a VOP header (VOP header in the figure) including header information about the VOP are attached.

  The VOP code string is further divided into code strings for each video packet including macroblock data (MBdata in the figure). At the head of the image code string of each video packet, a uniquely decodable synchronization code called a resync marker (RM in the figure) and a video packet header (Video packet header in the figure) following the code are attached. However, since a VOP start code and a VOP header are attached to the first video packet of the VOP, a resync marker (RM) and a video packet header (Video packet header) are not added here.

  FIG. 14B shows an example of header information included in the VOP header. In the figure, modulo time base (MTB) and VOP time increment (VTI) are information indicating the time of the VOP. This time information is used to define the timing for decoding and displaying the frame of the VOP.

  Here, the relationship between the time of the VOP, the modulo time base (MTB), and the VOP time increment (VTI) will be described with reference to FIG. The VOP time increment is information indicating the time with millisecond precision, and contains a value obtained by taking the remainder of the VOP time in 1000 milliseconds (1 second). The modulo time base is information indicating the time with a second precision. If the second is the same as the VOP encoded immediately before, the modulo time base is "0", and if the second is different, the difference is the difference value.

  For example, as shown in FIG. 15, when the time (millisecond) of the VOP is 0, 33, 700, 1000, 1300, 1833, 2067, the VOP time increment is a remainder of 1000, that is, 0, 33, 700, 0, 300, 833, 67. Further, the reason that the modulo time base is 1 is that the value obtained by dividing the time by 1000 and rounding down the decimal point (in the example of the figure, 0, 0, 0, 1, 1, 1, 2) is the value of the immediately preceding VOP. In other words, it becomes “1” in the VOP corresponding to time = 0, 1000, 2067, and becomes “0” in other VOPs. In the modulo time base, encoding using a variable length code may be performed. For example, when the modulo time base is 0, 1, 2,..., The variable length codes can be used in association with “1,” “01,” “001,” respectively.

  The VOP prediction mode (VPT in the figure) of FIG. 14B is information indicating the prediction encoding mode (I, B, P) of the entire VOP. The VOP quantization parameter (PQ in the figure) is information indicating the quantization step width used for encoding the VOP. However, when a VOP is divided into a plurality of video packets and encoded, the quantization step width can be changed for each video packet, and thus the information may be information indicating the quantization step width of the first video packet.

  FIGS. 14C and 14D show examples of information included in a video packet header (Video packet header), which is header information added to a video packet. The macroblock number (MBA in the figure) is information indicating the number of the first macroblock of the video packet. The video packet quantization parameter (SQ in the figure) is information indicating the quantization step width of the video packet. The header extension code (HEC in the figure) is a flag indicating whether important information to be duplicated (multiplexed) is added to the video packet header. When HEC is “0”, no important information is added as shown in FIG. 14C, and when HEC is “1”, important information is added as shown in FIG. 14D. In the example of FIG. 14D, in order to be able to restore the time information of the image frame, as important information, a modulo time base (MTB in the figure) indicating the time of the VOP and a VOP time increment (VTI in the figure) are included. It is added as it is.

  FIG. 16 is a block diagram showing a configuration of a decoding device corresponding to the image coded sequence of FIG. Here, portions corresponding to those of the decoding apparatus of FIG. 2 are denoted by the same reference numerals, and only the differences will be described. The decoding device in FIG. 16 includes a VOP header decoding circuit 601, a video packet header decoding circuit 602, a time decoding circuit 603, and a temporary storage circuit 621 in addition to the configuration of the decoding device in FIG.

  When the VOP start code is detected by the synchronization detection circuit 122, a signal indicating that the VOP start code is detected is notified to the decoding circuit 124. In response to the notification, the decoding circuit 124 sends a code string including the VOP header subsequent to the VOP start code, that is, the first video packet to the VOP header decoding circuit 601 where the VOP header is decoded. The VOP header decoding circuit 601 decodes the time, VOP encoding mode, and VOP quantization parameter included in the VOP header. Among them, regarding the time, the modulo time base and the VOP time increment are sent to the time decoding circuit 603, where the time is decoded.

  The time decoding circuit 603 decodes the transmitted modulo time base and VOP time increment, and checks whether there is any error. The error is checked by verifying that the time decoded from the modulo time base and the VOP time increment is a time that can actually exist. For example, when the encoded image signal is an NTSC signal, the frame rate is 30 Hz, so that the time should take a multiple of 1/30 seconds (= 33 milliseconds). Therefore, if the decoding time is not a multiple of 1/30 second, it is determined that there is a transmission path error in the modulo time base and the VOP time increment. At the same time, in the case of a PAL signal, it is possible to check whether the signal is a multiple of 1/25 seconds.

  As the reference value for such an error check, a value predetermined by the encoding device and the decoding device may be prepared for each type of image signal (PAL, NTSC, CIF, etc.), or ( Information indicating this may be put in a code string of system information (not shown), or may be put in a part of an image code string.

  When the decoding of the time and the error check in the time decoding circuit 603 are completed, the VOP header decoding circuit 601 decodes the signal indicating that there is an error, and decodes the signal indicating that there is no error. A signal is sent indicating the time of the event. When there is no error in the time, the VOP header decoding circuit 601 stores information indicating the time in the temporary storage circuit 621, and sends the information together with other information to the decoding circuit 124. On the other hand, when there is an error in the time, the code string of the first video packet including the VOP header is discarded, and decoding of the next video packet is started.

  When the resync marker (RM) is detected by the synchronization detection circuit 122, a signal indicating that they are detected is transmitted to the decoding circuit 124. In response, the decoding circuit 124 sends a code string including a video packet header following the resync marker (RM), that is, a certain second or subsequent video packet, to the video packet header decoding circuit 601, where decoding of the video packet header is performed. Let it run. The video packet header decoding circuit 602 decodes a macro block number (MBA), a video packet quantization parameter (SQ), and a header extension code (HEC) included in the video packet header.

  If the header extension code (HEC) is “1”, the subsequent modulo time base and VOP time increment are sent to the time decoding circuit 603, where the time is decoded. The time decoding circuit 603 decodes the transmitted modulo time base and VOP time increment and checks whether there is any error, as in the case of decoding the VOP header. When the time decoding and error checking by the time decoding circuit 603 are completed, the video packet header decoding circuit 602 outputs a signal indicating that there is an error, and decodes the signal if it is determined that there is no error. A signal is sent indicating the time at which it was performed. When there is an error in the time, the video header decoding circuit 602 discards the code string of the video packet including the VOP header and decodes the next video packet.

  On the other hand, if there is no error in the time, the time is compared with the time stored in the first temporary storage circuit 621, that is, the time obtained in the video packet decoded immediately before, and the current video packet is included. Is determined. If the time is the same, it is determined that the video packet is included in the same VOP as the video packet decoded immediately before, and a signal indicating the information of the decoded video packet header is sent to the decoding circuit 124. Perform decryption. On the other hand, when the decoded time is different from the time stored in the temporary storage circuit 621, it is determined that the video packet to be decoded is included in a different VOP from the video packet decoded immediately before. I do. In this case, the decoding time is recorded in the temporary storage circuit 621, and the decoding circuit 124 has a VOP area between the immediately preceding video packet and this video packet, and decodes this video packet as a new VOP. Then, a VOP division signal indicating that the video packet is to be transmitted, a decoding time, and a signal indicating information of the decoded video packet header are transmitted. The decoding circuit 124 receives the VOP division signal, performs VOP decoding end processing on the assumption that the VOP has ended with the video packet decoded immediately before, and regards the video packet to be decoded from now on as the first video packet of the next VOP. To perform VOP decoding start processing, and subsequently decode video packets.

  By performing such processing, even if the VOP start code and the VOP header are lost due to an error, the VOP boundary can be determined based on the time information of the video packet header, and the correct decoding time can be obtained. The quality of the decoded image is improved.

  FIG. 17 shows a second example of the VOP header and the video packet header. The difference from the example of FIG. 14 is that a CRC check bit for checking whether or not an error is included in the information included in the header is added.

  FIG. 17A shows a VOP header. In the drawing, CW1 is a check bit for performing a CRC check on a modulo time base, a VOP time increment, a VOP encoding mode, and a VOP quantization parameter included in the VOP header.

  FIGS. 17B and 17C show a video packet header. In the figure, CW2 is a check bit for performing a CRC check on a macroblock number, a video packet quantization parameter, and a header extension code. Also, CW3 is HEC = "1", that is, a check bit that exists only in the video packet header to which important information has been added, and that performs a CRC check on important information, that is, modulo time base and VOP time increment.

  FIG. 18 shows the configuration of a decoding device corresponding to the image code string in FIG. The same reference numerals are given to portions corresponding to the decoding device in FIG. 16 and only the difference will be described. The difference is that a CRC determination circuit 605 is added.

  The VOP header decoding circuit 601 decodes information included in the VOP header and performs a CRC check of the VOP header using the CRC check bit CW1. If it is determined by the CRC check that there is an error, the VOP header and the video packet including the VOP header are discarded, and decoding of the next video packet is started.

  The video packet header decoding circuit 602 decodes information included in the video packet header and performs a CRC check on the video packet header using the CRC check bit CW2. If it is determined by the CRC check that there is an error, the video packet header and the video packet including the header are discarded, and the process proceeds to the decoding of the next video packet. If it is determined that there is no error, and if the decoded header extension code HEC is “1”, the subsequent added important information (MTB and VTI) is decoded. Then, it is checked whether or not there is an error in the important information by using the CRC check bit CW3. If there is no error, comparison with the time information in the VOP header and other video packet headers, VOP division processing, etc. are performed as in the case of the decoding apparatus in FIG.

  As described above, in the second embodiment, since information indicating the time of an image frame is added to a video packet in the frame, the video packet is lost even if the time information included in the VOP header is lost due to an error. Since the correct time can be decoded based on the important information in the header, the decoding device can reproduce and display the image at the correct time.

  In addition, by comparing the time information of the video packet header with the time information of the VOP header or another video packet header and performing the VOP boundary determination, the VOP boundary can be correctly decoded even if the VOP start code is lost due to an error. Thus, the quality of the decoded image is improved.

  In the above example, whether or not important information is included is indicated by a header extension code (HEC) for each video packet. For example, HEC = "1" may be set in all video packet headers to put important information, or HEC = "1" may be set only in some video packets. By controlling the number of video packets containing important information according to the state of the transmission path error, important information can be efficiently protected with a small overhead.

  For example, when the decoding time is represented by using the modulo time base (MTB) and the VOP time increment (VTI) as in the above-described example, when the time information cannot be correctly decoded when MTB = 0, the decoding time Is less than 1 second, but if the MTB is not correctly decoded in a VOP whose MTB is other than 0, a large error in the decoding time will occur in the decoding time in subsequent VOPs. For this reason, in a VOP having an MTB of 0, HEC = "0" in all video packets or HEC = "1" in only a small number of video packets, and in a VOP having an MTB other than 1, HEC is used in all or many video packets. = “1” so that the MTB is correctly decoded.

  In the second embodiment, an example in which information indicating a time (modulo time base, VOP time increment) is used as important information to be duplicated is described. However, other than this information, for example, information indicating an encoding mode, quantum It may be duplicated together with information representing the conversion parameter, information on motion compensation, motion vector information, and the like.

  When encoding by switching a plurality of VOP prediction modes (for example, intra-frame prediction VOP (I-VOP), forward prediction VOP (P-VOP), forward-backward prediction VOP (B-VOP)), for each VOP If the information on the VOP prediction mode cannot be correctly decoded, the VOP cannot be decoded. By including the VOP prediction mode in the video packet header as duplicated information, even if the VOP prediction mode of the VOP header is lost due to an error, the VOP prediction mode information included in the duplicated information in the video packet header is used. It is possible to decode the VOP. Hereinafter, such an example will be described.

  FIG. 27 illustrates a third example of the video packet header according to the second embodiment. The image code string and VOP header of one entire frame (VOP) are the same as those in FIGS. 14A and 14B, respectively. FIGS. 27A and 27B show video packet headers when the header extension codes HEC = "0" and HEC = "1", respectively. The difference from the video packet header of FIG. 14 is that when HEC = "1", a VOP prediction mode (VPT in the figure) is included in addition to information (MTB, VTI in the figure) indicating the time.

  The decoding device corresponding to the image code string in FIG. 27 has the same overall configuration as that in FIG. However, the operation of the video packet header decoding circuit 602 is different. Another difference is that not only time information (modulo time base, VOP time increment) but also VOP prediction mode information (VPT) is recorded in the temporary storage circuit 621. Hereinafter, the operation of the decoding circuit will be described focusing on this difference.

  When the VOP start code is detected by the synchronization detection circuit 122, a signal indicating that the VOP start code is detected is transmitted to the decoding circuit 124. In response, the decoding circuit 124 sends a code string including a VOP header following the VOP start code to the VOP header decoding circuit 601, and the VOP header is decoded. The VOP header decoding circuit 601 decodes time information (MTB, VTI), VOP coding mode information (VPT), and VOP quantization parameter (PQ) included in the VOP header. Among them, as for the time, a modulo time base (MTB) and a VOP time increment (VTI) are sent to the time decoding circuit 603, where the time is decoded.

  The time decoding circuit 603 first decodes the sent modulo time base and VOP time increment, and checks whether there is any error. The error is checked by verifying that the time decoded from the modulo time base and the VOP time increment is a time that can actually exist. For example, when the encoded image signal is an NTSC signal, the frame rate is 30 Hz, so that the time should take a multiple of 1/30 seconds (= 33 milliseconds). Therefore, when the decoding time is not a multiple of 1/30 seconds, it is determined that there is a transmission path error in the modulo time base and the VOP time increment. Similarly, in the case of a PAL signal, it is possible to check whether it is a multiple of 1/25 seconds.

  As the type of the image signal (PAL, NTSC, CIF, etc.), a value predetermined by an encoding device and a decoding device may be used, or information indicating this may be put in a code string of system information (not shown). It may be shown or a part of the image code sequence may be present.

  When the decoding of the time and the error check in the time decoding circuit 603 are completed, the VOP header decoding circuit 601 decodes a signal indicating that there is an error and decodes a signal indicating that there is no error. A signal indicating the time is sent. If there is no error in the time, the VOP header decoding circuit 601 further decodes VOP prediction mode information (VPT). If there is no error in the VOP prediction mode information, the time information and the VOP prediction mode information are stored in the temporary storage circuit 621 and sent to the decoding circuit 124 together with other information. On the other hand, when there is an error in the time information or the VOP prediction mode information included in the VOP header, the code string of the video packet including the VOP header is discarded, and the next video packet is decoded.

  When the resync marker is detected by the synchronization detection circuit 122, a signal indicating that they are detected is transmitted to the decoding circuit 124. In response, the decoding circuit 124 sends a code string including a video packet header following the resync marker to the video packet header decoding circuit 601 to decode the video packet header. The video packet header decoding circuit 601 decodes a macroblock number, a video packet quantization parameter, and a header extension code included in the video packet header.

  If the header extension code HEC = “1”, the subsequent modulo time base and VOP time increment are sent to the time decoding circuit 603, where the time is decoded. The time decoding circuit 603 decodes the transmitted modulo time base and VOP time increment and checks whether there is any error, as in the case of decoding the VOP header. When the decoding of the time and the error check in the time decoding circuit 603 are completed, a signal indicating that the video packet header decoding circuit 602 has an error is decoded when it is determined that there is no error. A signal is sent indicating the time of the event. When there is an error in the time, the video packet header decoding circuit 602 discards the code string of the video packet including the video packet header and decodes the next video packet.

  On the other hand, if there is no error in the time, the VOP prediction mode information subsequent to the time information is further decoded. If there is no decoding error in the VOP prediction mode information, the decoded time information is compared with the time stored in the first temporary storage circuit 621, and the VOP including the video packet is compared. Is determined. If the time is the same, it is determined that the video packet is included in the same VOP as the video packet decoded immediately before, and a signal indicating the information of the decoded video packet header is sent to the decoding circuit 124. Perform decryption. On the other hand, when the decoded time is different from the time stored in the temporary storage circuit 621, it is determined that the video packet to be decoded is included in a different VOP from the video packet decoded immediately before. I do. In this case, the decoded time information and the VOP prediction mode information are recorded in the temporary storage circuit 621, and the VOP division signal indicating that this video packet is the first of the VOP, the decoded time, Then, a signal indicating the information of the decoded video packet header is sent. The decoding circuit 124 receives the VOP division signal, performs VOP decoding end processing on the assumption that the VOP has ended with the video packet decoded immediately before, and regards the video packet to be decoded from now on as the first video packet of the next VOP. To perform VOP decoding start processing, and subsequently decode video packets.

  If the VOP prediction mode information included in the video packet header is different from the VOP prediction mode information recorded in the temporary storage circuit 621, the video packet is included in the VOP prediction mode included in the video packet header. May be used to perform the decoding process. Accordingly, even when the VOP prediction mode information included in the VOP header cannot be correctly decoded, the video packet can be decoded.

  By performing such processing, even if the VOP start code and the VOP header are lost due to an error, the VOP boundary and the VOP prediction mode can be correctly identified by the time information and the VOP prediction mode information of the video packet header. Quality is improved.

  When decoding a VOP header or a video packet header, if there is error check information (CRC, stuffing bit, etc.) in the image code string (not shown), or a code string from the transmission path / storage medium, If the receiving circuit or the demultiplexing circuit that separates the received code string into an image code string and a sound code string has a function of determining that there is an error in the code string, the error is determined by these. It may be determined whether there is an error in the decoded VOP header or video packet header using the result. If it is determined that there is an error in the decoded information, the information is not used for image decoding. Also, a video packet including information determined to have an error may be discarded without decoding.

  FIG. 28 shows a fourth example of the video packet header according to the second embodiment. The image code string and the VOP header of one entire frame (VOP) are the same as those in FIGS. 14A and 17A, respectively. FIGS. 28A and 28B show video packet headers when HEC = "0" and HEC = "1", respectively. Compared with the video packet headers of FIGS. 17B and 17C, when HEC = "1", a VOP prediction mode (VPT in the figure) is included in addition to the information (MTB and VTI in the figure) indicating the time. Are different.

  The decoding device corresponding to the image code string in FIG. 28 has the same overall configuration as that in FIG. However, the operation of the video packet header decoding circuit 602 is different. Another difference is that not only time information (modulo time base, VOP time increment) but also VOP prediction mode information (VPT) is recorded in the temporary storage circuit 621. Hereinafter, the operation of the decoding circuit will be described focusing on this difference.

  The VOP header decoding circuit 601 decodes information included in the VOP header and performs a CRC check of the VOP header using the CRC check bit CW1. If it is determined by the CRC check that there is an error, the VOP header and the video packet including the VOP header are discarded, and decoding of the next video packet is started.

  The video packet header decoding circuit 602 decodes information included in the video packet header and performs a CRC check on the video packet header using the CRC check bit CW2. If it is determined by the CRC check that there is an error, the video packet header and the video packet including the header are discarded, and the process proceeds to the decoding of the next video packet. If it is determined that there is no error, and if the decoded header extension code HEC is 1, the subsequent duplicated important information (MTB, VTI and VPT in the figure) is decoded. Then, using the CRC check bit CW3, it is checked whether the duplicated important information contains an error. If there is no error, comparison with the VOP header or time information in other video packet headers, VOP division processing, etc. are performed in the same manner as in the decoding apparatus of FIG.

  As described above, since the information representing the time is included in the important information, the correct time can be decoded by the important information in the video packet header even if the time information included in the VOP header is lost due to an error. Can reproduce and display an image at a correct time. In addition, by comparing the time information of the video packet header with the time information of the VOP header or another video packet header and performing the VOP boundary determination, the VOP boundary can be correctly decoded even if the VOP start code is lost due to an error. Thus, the quality of the decoded image is improved.

  Also, by including the VOP prediction mode in the video packet header as duplicated information, even if the VOP prediction mode of the VOP header is lost due to an error, the VOP prediction mode information included in the duplicated information in the video packet header is retained. It is possible to decode the VOP based on it.

  In the second embodiment, a bit called a marker bit is added to the VOP header and the video packet header so that the same pattern as a synchronization code (Picture start code, VOP start code, Resync marker, etc.) is not generated. Good.

  FIG. 31 shows an example in which marker bits are added to the code string in FIG. In FIG. 31, “marker” after VOP time increment (VTI) is a marker bit, and has a predetermined bit value (for example, “1”).

  FIG. 32 shows a comparison between a video packet header without marker bits and a video packet header with marker bits. As shown in FIG. 32A, the resync marker, which is one of the synchronization codes, is a 17-bit code word having a bit pattern of “000000000000000001”. VOP time increment (VTI) is a code word having an arbitrary value of 10 bits, and MTB is a variable length code in which the last bit is 0.

  As shown in FIG. 32B, when there is no marker bit, when the VTI is a continuous pattern of 0, the same bit pattern as that of the resync marker occurs. In the example shown in FIG. 32B, the same pattern as the resync marker has been generated in the bit string of “0” of the MTB, “000000000000” of the VTI, and “000001” following the bit string.

  On the other hand, by adding a marker bit “1” after the VTI as shown in FIG. 32 (c), the number of consecutive 0 bits in the video packet header can be up to 11 bits (0 in the last 1 bit of the MTB). And the VTI “000000000000”), the same bit pattern as that of the resync marker is not generated.

  Since the marker bit has a predetermined bit value (“1” in the example of FIG. 32), the decoding device determines whether the marker bit has the predetermined value, thereby determining whether the VOP header, the video It may be determined whether there is a transmission path error in the packet header.

  17, 27, 28, etc. shown in the second embodiment can be used by adding marker bits.

  Further, such a structure of the code string can be applied to a case where a slice layer is used. FIG. 33 shows another example of a code string using the slice structure in the first embodiment.

  In FIG. 33, SSC is a slice synchronization code, EPB is a bit having a bit value “1” added so that a portion other than a synchronization code (for example, SSC) does not have the same bit pattern as the synchronization code, and MBA is a slice synchronization code. Information indicating the number of the first macroblock, SQUANTT is a quantization parameter used in the slice, and GFID is information included in the picture header or information indicating a part thereof. When the synchronization code SSC is aligned with the byte position in the code string, a stuffing bit SSTUF is added before the SSC. Macroblock Data is information on each macroblock.

  TR is duplicated important information, and indicates time information (Temporal Reference). TRI is a 1-bit flag indicating whether TR has been added or not. When TRI = 1, TR is added.

  Next, a third embodiment of the present invention will be described.

  FIG. 19 shows the overall configuration of a video / audio coding apparatus according to the third embodiment. The moving image signal 101A and the audio signal 102A to be compression-encoded are input to the moving image encoding device 111A and the audio encoding device 112A, respectively, and compressed to output the moving image code sequence 121A and the audio code sequence 122A, respectively. The configurations of the moving picture coding apparatus and the speech coding apparatus are detailed in the literature (edited by Hiroshi Yasuda, “International Standard for Multimedia Coding”, Maruzen (1994)) and so on, and are omitted here.

  The moving image code string 121A and the audio code string 122A are multiplexed together with the data code string 103A by the multiplexer 130A, and a multiplexed code string 135A is output.

  FIG. 20 is a diagram showing the overall configuration of a video / audio decoding device corresponding to the video / audio coding device of FIG. The multiplexed code sequence 185A from the video / audio coding device is separated by the demultiplexing device 180A, and a video code sequence 171A, a voice code sequence 172A, and a data code sequence 173A are output. The video code sequence 171A and the audio code sequence 172A are input to the video decoding device 161A and the audio decoding device 162A, respectively, where they are decoded to output a reproduced video signal 151A and a reproduced audio signal 152A. .

  FIGS. 21A and 21B show two examples of the video code sequence 121A. Encoding in the moving image encoding device 111A is performed in units of pictures (frames, VOPs), and a moving image code sequence 121A is created. The picture is further divided into small areas called macroblocks and encoded.

  A video code sequence of one picture starts with a picture start code (PSC) 201A (also referred to as a VOP start code), which is a uniquely decodable code indicating the start position of a picture.

  A picture header (PH) 202A (also referred to as a VOP header) follows the picture start code 201A. The picture header 202A includes a PTR (Picture Temporal Reference) 221A indicating the temporal position of the picture, a picture encoding mode (PCM) 222A indicating the encoding mode of the entire picture, and a picture quantization step size (PQ) 223A. ing. Encoded data 203A of each macroblock follows the picture header 202A.

  FIG. 21B shows an example in which encoding is performed for each slice in which a plurality of macro blocks are put together.

  In the code string of each slice, a resynchronization marker (RM) 210A and a slice header (SH) 211A, which are uniquely decodable codes indicating the start position of the slice, follow, and macroblock data (MB) 203A of each macroblock. Followed by The slice header 211A includes an SMBN (Slice Macroblock Number) 231A indicating the number of the first macroblock of the slice, and a quantization step size (SQ) 232A.

  The resynchronization marker 210A and the slice header 211A may be added for each predetermined fixed bit, or may be added to a specific position in an image frame. When coding having a slice structure is performed as described above, even if an error is mixed in a video code sequence, resynchronization can be performed with the resynchronization marker 210A that can be uniquely decoded, and the error propagates. Since the range can be included in the slice, the quality of a reproduced image when a transmission path error occurs is improved.

  FIG. 22 is a diagram illustrating an example of the multiplexed code string 135A multiplexed by the multiplexer. The multiplexed code sequence 135A is composed of a plurality of multiplexed packets in which a video code sequence (VIDEO), an audio (audio) code sequence (SPEECH), data, and a control information code sequence (DATA) are multiplexed for each predetermined size. It is composed of In FIG. 22, the sections denoted by 301A, 302A, and 303A are one multiplexed packet. In this case, all multiplexed packets may be fixed-length packets having the same length (number of bits), or variable-length packets having different lengths for each multiplexed packet.

  Each multiplexed packet first has a multiplexing start code (MSC) 310A indicating the start position of the multiplexed packet, followed by multiplexing headers (MH) 311A, 312A, and 313A. This is followed by a multiplexed payload (321A, 322A, 323A in the figure) obtained by multiplexing the video code sequence 121A, the audio code sequence 122A, and the data code sequence 103A in FIG.

  FIGS. 23A and 23B are diagrams illustrating a first example of information included in the multiplex header (MH) 311A. In the figure, a multiplexing code (MC) 351A indicates how a moving image code string (Video), an audio code string (Speech), and a data code string (Data) are multiplexed in the multiplexed payload 321A. The information is shown. If a transmission error occurs in the multiplexed code information (MC), it becomes impossible to know how the multiplexing is performed. become unable. Therefore, even the video decoding device 161A and the audio decoding device 162A cannot perform correct decoding, and the quality of the reproduced video signal and the audio signal is deteriorated.

  In order to avoid this, each of the multiplexed headers (MH) performs strong error protection using an error check code and an error correction code. In FIG. 23, 353A (CRC) is a CRC error check bit, and 354A (FEC) is a check bit of an error correction code.

  In the third embodiment, the multiplexed header (MH) of the multiplexed packet including the video code sequence (Video) includes the video header information (VHD) 352A of the video code sequence (Video) as well as the multiplexed code information (MH). MC). In the example of FIG. 23, the multiplex header (MH) including the video header information (VHD) 352A is MH1 (311A) and MH2 (312A). The video header information (VHD) 352A is important information such as a coding mode of an entire picture (frame) in moving picture coding, in which an error causes significant deterioration in a reproduced picture. For example, when the encoded video sequence is as shown in FIG. 21, the picture header 202A, the slice header 211A, or a part of the information is put in the multiplexed header as video header information (VHD) 352A.

  As described above, important information such as a picture header in video coding is inserted into the multiplex header, and an error correction code and an error detection code are generated together with the multiplex code (MC), thereby providing strong error protection. Performing is a feature of the third embodiment. As a result, resistance to transmission path errors is improved as compared with a conventional moving picture coding apparatus that does not perform error protection on important information.

  FIG. 24 is a diagram illustrating a second example of the multiplex header (MH). The same reference numerals are given to the information corresponding to the first example shown in FIG. 23, and only the difference will be described. A multiplexed packet including a moving image code string (Video) is included in a multiplexed header. The first example is different from the first example in that a picture pointer (ALP) 451A indicating the position of a picture or slice boundary in a moving image code string is included in addition to the video header information (VHD) 352A.

  If there is no picture pointer (ALP) 451A, it is necessary to demultiplex the moving image code string by the demultiplexing device 180A and then detect the picture or slice boundary by the picture start code or the resynchronization marker by the moving image decoding device 161A. is there. On the other hand, when the picture pointer (ALP) 451A is included in the multiplexed header, a picture or slice boundary can be detected by the picture pointer. Since the picture pointer is strongly protected from error in the multiplexed header, the probability of correctly detecting a picture boundary or a slice boundary is improved, and the quality of a reproduced image is improved.

  Further, the video header information (VHD) 352A may include all information included in the picture header and the slice header, or may include only a part of the information.

  FIG. 25 shows, as multiplexed codes (MC1) 611A and (MC2) 621A, in each of the multiplexed headers of the multiplexed packets 601A and 601B including the video code sequence (Video), as video header information. , Picture Time Reference (PTR1) 612A, and (PTR2) 622A.

  In the figure, in the multiplexed payload of the multiplexed packet 601A, the last slice (Slice N) 613A of the code string of the picture of PTR = 1, and the picture start code (PSC) 614A of the picture of PTR = 2 that follows. , PTR = 2, a Picture Time Reference (PTR2) 615A, a picture encoding mode (PCM2) 616A, and a first half (Slice 1) 617A of the first slice of the code sequence of the picture with PTR = 2. In the payload of the multiplexed packet 602A, the latter half (Slice 1) 623A of the first slice of the code string of the picture with PTR = 2, the resynchronization marker (RM) 624A of the second slice, and the slice header (SH2) 625A , PTR = 2 are included in the second slice (Slice 2) 626A of the code sequence of the picture.

  The multiplexed header (MH1) of the multiplexed packet 601A includes the PTR 612A of the PTR = 1 picture having the last part of the code string in the multiplexed packet 601A, and the error correction together with the multiplexed code (MC1) 611A. , Error detection with a detection code (CRC, FEC). Therefore, even if the PTR (615A) included in the video coded stream of the multiplexed payload cannot be decoded correctly due to an error, the PTR (612A) in the multiplexed header can be correctly decoded, and a correct PTR can be obtained. Therefore, the decoded picture can be displayed at the correct time.

  Further, in the video coding method using the slice structure, if the PTR is inserted in the slice start code (resynchronization marker) and the video header information 352A of the multiplexed packet including the slice header, the picture start code may be incorrectly set due to an error. Even if decoding is not possible, a picture boundary can be determined by this PTR. For example, even if the picture start code (PSC) 614A or PTR 615A is missing, the multiplexed header of the next multiplexed packet contains PTR 622A. The included PTRs (for example, PTR 612A) are compared, and if they are not equal, it is determined that the multiplexed packet 601A has a picture boundary. In this case, it is possible to correctly decode the first slice (from slice 2 of 624A in the example in the figure) in which the resynchronization marker is included in the multiplexed packet 602A.

  When using an encoding method in which the picture encoding mode changes frequently (for example, an encoding method using a B picture), the multiplex header may include the picture encoding mode.

  FIG. 26 shows a third example of the multiplexed code string. In this multiplexed code string, one picture or slice is included in each multiplexed packet 701A, 702A, 703A, and each picture header (PH1) 712A and slice header (SH2) 722A are included in the multiplexed headers 751A, 752A. Error protection is performed together with the multiplexing codes (MC1) 711A and (MC2) 721A. In this way, if the pictures and slices for video coding are aligned with the multiplexed packet, each multiplexed packet can be uniquely identified as being the start position of a picture or slice. There is no need to detect the picture start code or the resynchronization marker from the column again, and the processing amount is reduced. Furthermore, if the multiplexed start code is a code having a strong resistance to transmission path errors, the probability that the picture or slice cannot be decoded because the start position of the picture or slice cannot be specified correctly is reduced.

  In the third embodiment, an example in which one image / audio signal is encoded and decoded has been described. However, a plurality of image / audio signals are encoded and encoded using a plurality of image / audio signal encoding devices. The present invention can be similarly applied to the case of multiplexing and separating / decoding a plurality of image / audio signals using a plurality of image / audio signal decoding devices. In this case, the video header information included in the multiplexed header information may include information for identifying a plurality of image signals.

  In addition, by generating the transmission code sequence by appropriately combining the third embodiment with the first and second embodiments, it is possible to transmit encoded information with higher reliability. In the third embodiment, a PTR (Picture Time Reference) representing time can be handled as a modulo time base and a VOP time increment similar to the second embodiment. By using in this manner, a check using the regularity of modulo time base and VOP time increment may be performed in the same manner as described in the second embodiment.

  Next, a specific example regarding a medium for storing information according to the present invention will be described.

  FIG. 29 is a diagram showing a system for reproducing an image signal using the recording medium 810 in which the image code string output from the encoding device according to the present invention is stored. The recording medium 810 stores a code string including an image code string encoded by the image encoding device according to the present invention. A decoding device 820 reproduces an image signal from the code string stored in the storage medium 810, and an image information output device 830 outputs a reproduced image. Here, the image information output device refers to, for example, a display or the like. Alternatively, the reproduced image signal may be recorded on a storage medium (not shown) or transmitted to another device or system via a transmission path (not shown).

  In this system having such a configuration, a code string in the format described in each of the above-described embodiments is stored in the storage medium 810. This code string is characterized in that part of VOP (also called picture or frame) header information is recorded as duplication information in part of a video packet (or slice, GOB, etc.) header. The decoding device 820 reproduces an image signal from the code string stored in the storage medium 810. That is, the decoding device 820 reads a code string from the storage medium 810 via the signal line 801 and generates a reproduced image according to the procedure shown in FIG.

  Hereinafter, the content of the processing in the decoding device 820 will be described with reference to FIG.

  The image code sequence is sequentially read from the storage medium 810, and a synchronization code is detected first (step S11). If the detected synchronization code is the VOP start code (YES in step S12), a process of outputting the VOP (frame) decoded immediately before to the image information output device is performed (step S13). Then, the VOP header (VOP header in FIG. 29) subsequent to the VOP start code in the image code string is decoded (step S14). If the VOP header can be correctly decoded (YES in step S15), the information recorded in the temporary storage circuit in the decoding device is replaced with the decoded VOP header information (time information, VOP prediction mode, etc.) (step S15). S16). Then, the macroblock data (MB data in FIG. 29) following the VOP header is decoded, and the video packet is decoded (step S17).

  If the detected synchronization code is a resync marker (YES in step S18), a video packet header (macroblock number (MBA), video packet quantization parameter (SQ), header extension code HEC) following the resync marker is used. ) Is decrypted (step S19). If the header extension code HEC in the video packet header is "0" (NO in step S20), the video packet is decoded (step S17). If the header extension code HEC = "1" (YES in step S20), the subsequent duplication information (DUPH in FIG. 29) is decoded (step S21). If the duplicated information has been correctly decoded (YES in step S22), the duplicated information is compared with the information stored in the temporary storage circuit (step S23). If the comparison results are equal (NO in step S23), the macroblock data (MBdata in FIG. 29) following the video packet header is decoded, and the video packet is decoded (step S17). If the comparison results are not equal (YES in step S23), the process determines that the video packet belongs to a VOP different from the VOP decoded immediately before, and outputs the VOP decoded immediately before to the image information output device. (Step S24), and replace the information recorded in the temporary storage device with the decrypted duplicated information (Step S25). Further, the video packet is decoded (step S17).

  As described above, a series of processes starting from the detection of the synchronization code shown in FIG. 30 is repeated while sequentially reading the image code sequence recorded on the storage medium 810 to reproduce the moving image signal.

  Instead of recording the image code string directly on the storage medium, a code string obtained by multiplexing the audio signal or the audio signal with the code string, data, control information, or the like is recorded on the storage medium. May be. In this case, before the information recorded in the storage medium is decoded by the image decoding device 820, a process of demultiplexing the image code sequence and the audio / audio code sequence, data, and control information by the demultiplexing device is performed. The multiplexed image code sequence is decoded by the decoding device 820.

  Also, FIG. 29 shows an example in which information recorded in the storage medium 810 is transmitted to the decoding device 820 via the signal line 801. However, in addition to the signal line, a transmission path such as wired / wireless / infrared is used. The information may be transmitted via.

  As described above, according to the present invention, the important information is duplicated and recorded in the code string recorded on the storage medium. Even when an error occurs in a signal line or a transmission path for sending information recorded in a reproduction image to a reproduction image, a reproduction image with little deterioration can be reproduced.

  Next, a fourth embodiment of the present invention will be described.

  The overall configurations of the video / audio encoding device and the video / audio decoding device according to the present embodiment are the same as those in FIGS. 19 and 20, respectively. However, the operation of each unit is different from the third embodiment. Hereinafter, the difference will be mainly described.

  FIGS. 34 (a), (b) and (c) show three examples of the video code sequence 121A. Encoding in the moving image encoding device 111A is performed in units of VOPs (also referred to as pictures, frames, and fields) to create a moving image code sequence 121A. The picture is further divided into small areas called macroblocks and encoded.

  A moving image code sequence of one VOP starts with a VOP start code (VSC in the figure) (also referred to as a picture start code), which is a synchronous code that can be uniquely decoded. The VOP start code is followed by a VOP header (VH in the figure) (also called a picture header). The VOP header includes information indicating the time of the VOP, a VOP encoding mode, a VOP quantization step size, and the like. After the VOP header, encoded data of each macroblock follows.

  FIG. 34A shows an example in which encoding is performed by dividing the VOP into encoding units called video packets (also called slices or GOBs). A video packet is composed of one or a plurality of macroblocks (MBdata in the figure). For example, when performing video coding using prediction over a plurality of macroblocks, such as prediction from a motion vector of an adjacent macroblock of a motion vector, make sure that the effect of a transmission path error does not affect other video packets. Therefore, prediction may be performed only from macroblocks included in the same video packet.

  The code sequence of each video packet other than the first video packet of the VOP includes a resynchronization marker (RM) (also referred to as a slice start code and a GOB start code), which is a synchronization code that can be uniquely decoded, and a video packet header (VPH) ( (Also referred to as a slice header or GOB header), followed by data (MBdata) of each macroblock. The video packet header includes a macroblock number (or slice number or GOB number) indicating the position of the first macroblock of the video packet, a quantization step size of the video packet, and the like. Further, as described in the first and second embodiments, important information such as VOP header information may be included.

  FIG. 34 (b) shows a moving picture code sequence in which information on a prediction mode and a motion vector and information on a residual signal of motion compensated adaptive prediction or information on an orthogonal transform coefficient obtained by orthogonally transforming the signal (DCT or the like). FIG. 2 shows an example of a code string divided and coded. In the code string of each video packet, information on the prediction mode and the motion vector (Motion in the figure) is ahead (in the example of the figure, immediately after the video packet header or VOP header), and information on the prediction residual DCT coefficient ( In the figure, (Texture) is at the rear. The two types of information are separated by a motion marker (MM in the figure).

  FIG. 34 (c) shows an example of a code sequence of a moving image coding method in which information on the shape of an image to be coded is also coded. In the figure, Shape is information on shape information, and is located ahead of information (Motion) on a prediction mode and a motion vector in each video packet (in the example of the figure, immediately after a video packet header or a VOP header). The shape information (Shape) and the information (Motion) related to the prediction mode and the motion vector are separated by a shape marker (SM in the figure).

  In the moving picture coded sequence shown in FIG. 34, it is preferable that synchronization codes such as a start code and a resynchronization marker be aligned at a bit position that is an integral multiple of a certain number of bits. FIG. 35 shows an example in which the position of the VOP start code (VSC) and the position of the resynchronization marker (RM) at the beginning of each video packet are aligned to an integer multiple of N bits. By performing such processing, the position at which the decoding device detects the synchronization code can be reduced to 1 / N as compared with an encoding method in which the synchronization code is arranged at an arbitrary bit position. This simplifies the synchronization detection processing in the decoding device, and also reduces the phenomenon called similar synchronization in which the same bit pattern (pseudo synchronization code) as the synchronization code is generated due to a transmission path error and the synchronization code is erroneously detected. The probability can be suppressed to 1 / N, and the quality of a decoded image when a transmission path error occurs is improved.

  In order to align the positions of the synchronization codes in this manner, stuffing bits (stuffing bits in FIG. 35A) are inserted between the information immediately before the synchronization codes and the synchronization codes. FIG. 35B is a diagram showing an example of a stuffing bit code table when N = 8. This stuffing bit is different from a stuffing bit in which all bits are conventionally used in which all bits are "0", and is characterized in that the stuffing bit can be uniquely decoded from the reverse direction of the code string, and the length of the stuffing bit can be specified by a decoding device. . In the example of FIG. 35B, the first one of the stuffing bits is “0” and the other bits are “1”. Therefore, it is possible to determine the last bit of the stuffing bit, that is, the first bit of “0” as viewed from the bit immediately before the synchronization code in the reverse direction, as the first bit of the stuffing bit.

  As described above, since the position of the first bit of the stuffing bit can be specified, it is possible to easily detect that a transmission line error is mixed in the code string in the decoding device. If the code string is correctly decoded, the decoding end position of the data immediately before the stuffing bit and the start position of the stuffing bit should match. If the decoding end position is shifted from the start position of the stuffing bit, it may be determined that a transmission path error has occurred in the code string, and the code string may not be used for decoding.

  Further, when decoding in the reverse direction using a variable length code that can be decoded from the reverse direction of the code string, it is necessary to specify the start position of the backward decoding in the decoding device. The stuffing bit has a decoding start position immediately preceding the stuffing bit.However, since the length of a conventional stuffing bit, for example, in which all bits have the same bit value cannot be specified, the decoding device starts reverse decoding. I do not know the position. On the other hand, since the stuffing bit in FIG. 35 can specify the position of the first bit, it is possible to specify the start position of the backward decoding.

  Further, when the synchronization code is a code word including many “0” such as “000000000000000001”, the conventional stuffing bits of all “0” become the same bit pattern as the synchronization code due to the inclusion of an error. There is a problem that the probability is high and similar synchronization easily occurs. On the other hand, since the stuffing bits in FIG. 35 are all “1” except the first bit, the hamming distance is far from the synchronization code including many “0”, and the possibility of similar synchronization is low.

  As described above, by generating the stuffing bits according to a predetermined rule, the decoding / demultiplexing device checks the stuffing bits in the multiplexed code string against the generation rule, and if the generation rule is violated, When it is determined, it is possible to determine that an error has been mixed in the multiplexed code string. As a result, the decoding / demultiplexing device performs processing that does not cause large degradation in the demultiplexed and decoded signal, thereby improving the quality of the decoded signal when errors are mixed in the multiplexed code sequence. Can be improved.

  In addition to the VOP start code and the resynchronization marker, the motion marker (MM) and the shape marker (SM) are aligned at an integer multiple of a certain bit number, and a stuffing bit as shown in FIG. You may put in. As a result, error detection and reverse decoding can be performed on shape information, prediction mode, motion vector information, and the like.

  FIG. 36 is a diagram showing an example of the configuration of the multiplexer 130A. In the example of FIG. 36, the multiplexing process is performed in two stages called an adaptation layer 1031A (Adaptation Layer) and a multiplexing layer 1032A (Multiplex layer). The inputs to the adaptation layer 1031A are a video code sequence 121A, a voice code sequence 122A, and a data code sequence 103A. Outputs 1041A, 1042A, 1043A processed in the adaptation layer are input to the multiplexing layer 1032A. The output from the multiplexing layer becomes a multiplexed code string 135A.

  FIG. 37 shows an example of an output code string 1041A obtained by performing processing in the adaptation layer on the moving image code string 121A. The processing in the adaptation layer is performed for each unit of AL-SDU (also referred to as an access unit) obtained by dividing the video code sequence 121A into a certain unit. An output obtained by processing one AL-SDU in the adaptation layer is called an AL-PDU. FIG. 37 shows the configuration of one AL-PDU. Each AL-PDU has an AL-header. In the AL header, for example, information indicating the number and attribute of the AL-PDU, the mode of video encoding and multiplexing, and the like may be entered. The AL header is followed by an AL-SDU as an AL payload. Further, a check bit, such as a CRC check bit, for detecting whether or not a transmission path error has entered the AL-PDU may be added after the check bit.

  In the adaptation layer, similar processing is performed on the audio code sequence 121A and the data code sequence 103A, and AL-PDUs 1042A and 1043A for the audio code sequence and the data code sequence are output. However, the information to be put in the AL header, the length and presence or absence of the CRC check bit, and the like may be different from those of the AL-PDU 1041A for the moving image code string.

  The AL-PDUs 1041A, 1042A, 1043A created in the adaptation layer are multiplexed in the multiplexing layer. Multiplexing is performed for each unit called MUX-PDU. FIGS. 38A to 38C show examples of multiplexed MUX-PDUs. The MUX-PDU has a multiplexing synchronization code (MUX flag) and a multiplexing header (MUX header). The multiplex header may include information such as the type of output from the adaptation layer multiplexed in the MUX-PDU, the multiplexing method, and the length of the MUX-PDU.

  FIG. 38A shows an example in which one AL-PDU is put in one MUX-PDU.

  FIG. 38B shows an example in which one AL-PDU is divided into a plurality (two in the example in the figure) of MUX-PDUs. In this case, the multiplex header includes information indicating the number of the divided AL-PDU included in the MUX-PDU in the total number of one AL-PDU, and the first or last division of one AL-PDU. Information indicating an AL-PDU may be inserted.

  FIG. 38C shows an example in which a plurality of AL-PDUs are put in one MUX-PDU. In the example shown in the figure, an AL-PDU (Video AL-PDU) of a video code sequence and an AL-PDU (Audio AL-PDU) of an audio code sequence are combined and multiplexed. In this case, the multiplex header may include information indicating a boundary between a plurality of AL-PDUs included in the MUX-PDU. Alternatively, an identifier indicating the boundary may be attached to the boundary of the AL-PDU.

  As described above, the adaptation layer performs processing by dividing the code string into units called AL-SDUs or access units (access units). FIG. 39 is a diagram illustrating an example of a method of dividing a moving image code string in an adaptation layer.

  FIG. 39 shows an example in which one VOP is used as one access unit. FIGS. 39A to 39C correspond to the moving image code strings in FIGS. 34A to 34C, respectively.

  FIG. 40 shows an example in which one video packet is used as one access unit. FIGS. 40A to 40C correspond to the moving image code strings in FIGS. 34A to 34C, respectively.

  As shown in FIGS. 34 (b) and (c), when the video packet is further divided and encoded like shape information, motion vector information and DCT coefficient information, the access unit may be divided accordingly. Good. FIG. 41 shows such an example. FIGS. 41A and 41B correspond to the moving image code strings in FIGS. 34B and 34C, respectively. For each of the shape information (Shape), the information (Motion) related to the prediction mode and the motion vector, and the information (Texture) related to the residual signal and its DCT coefficient, a motion marker (MM) indicating the boundary and a shape marker (SM) are used as boundaries. To configure the access unit.

  As described above, when a multiplexing synchronization code indicating an MUX-PDU or AL-PDU boundary, an AL boundary identifier, or the like is added in the multiplexing layer, the start position of each access unit can be determined from this. . In this case, the synchronization code at the beginning of the access unit may be removed from the video code sequence. FIG. 42 shows an example in which one VOP is used as one access unit. In this case, the VOP start code at the head of the VOP may be removed. FIG. 43 shows an example in which one video packet is used as one access unit. In this case, the VOP start code and the resynchronization marker at the head of the video packet may be removed. FIG. 44 shows an example in which an access unit is configured for each of shape information (Shape), information (Motion) relating to a prediction mode and a motion vector, and information (Texture) relating to a residual signal and its DCT coefficient. A certain VOP start code, a resynchronization marker, a motion marker (MM) indicating a boundary between Shape, Motion, and Texture, and a shape marker (SM) may be removed.

  One access unit may contain one or more video packets as shown in FIG. In this case, as shown in FIG. 45B, only the VOP start code or the resynchronization marker at the beginning of the access unit may be removed. Similarly, the access unit may be composed of a plurality of video packets for the moving picture code strings in FIGS. 34 (b) and (c).

  As shown in FIGS. 34 (b) and (c), when a video packet is divided and encoded like Shape, Motion, and Texture, the access unit is collected by collecting Shape, Motion, and Texture of a plurality of video packets. You may comprise. FIG. 46 shows the result of performing such processing on the code string of FIG. 34B, and the access unit is configured by collecting Motion and Texture. The VOP header and the video packet header are added before Motion for each video packet.

  The unit constituting the access unit by collecting the Motion and the Texture may be a VOP unit or an arbitrary plurality of video packets.

  In such an access unit configuration, a synchronization code may be attached to the boundary between Motion and Texture of each video packet. FIG. 46B shows the case where the synchronization code (RM) is inserted at the boundary of Motion, and FIGS. 46C and 46D show the case where the synchronization code (RM) is inserted at the boundary of Motion and Texture. Further, in the example of FIG. 46D, a synchronization code (VSC) is also inserted at the head of each access unit. Different synchronization codes may be used for Motion and Texture. For example, a motion marker may be used in Motion, and a resynchronization marker may be used in Texture.

  It should be noted that the access unit can also be configured by collecting Shape, Motion, and Texture data for the moving image code sequence in FIG.

  As described above, access units having the same importance are collected from code strings having different importance such as Shape, Motion, and Texture, respectively, to constitute an access unit, and different error protection (for example, error correction) is provided for each access unit. (Code, error detection code, retransmission, etc.), it is possible to perform error protection according to the importance of each code string, and to improve the quality of a decoded image when a transmission path error occurs. In general, when a transmission path error enters the shape information (Shape), the mode information and the motion vector information (Motion), the quality of a decoded image is greatly deteriorated. Therefore, strong error protection such as using a strong error correction code for Shape and Motion may be performed. Conversely, the prediction residual signal (Texture) does not cause a great deterioration in image quality even if a transmission path error occurs, so that it is not necessary to perform the error protection so strongly, and the redundancy by the error correction code, the error detection code and the like is reduced. can do.

  In the above example in which the synchronization code of the moving image code string is removed, the synchronization code included in the moving image code string 121A may be removed by the multiplexer 130A, or the synchronization code may be removed in advance by the image encoder 111A. The moving image code string 121A may be passed to the multiplexer.

  In any of the examples of FIGS. 39 to 46, the length of each access unit may be set to an integral multiple (for example, byte unit) of a predetermined length. As shown in the example of FIG. 35, when a video code string includes a stuffing bit before a resynchronization marker or a start code and each video packet or each VOP is in N bits (for example, bytes), If the access unit includes the stuffing bit, the length of the access unit can be set to an integral multiple (eg, byte unit) of the determined length.

  If such processing is not performed on the video code string, a stuffing bit is inserted at the end of each access unit to make the length of the access unit an integer multiple (byte) of the determined length as shown in FIG. Unit). As the stuffing bit, for example, the stuffing bit in FIG. 35B may be used. In this case, similarly to the case where the stuffing bits are inserted in the moving image code string, it is also possible to detect an error mixed in the code string using the stuffing bits. In addition, a stuffing bit may be added to a voice or data code string in addition to a moving image code string to make the length of the access unit an integral multiple (such as a byte unit) of the determined length.

  In the multiplexing layer, when the same bit pattern as the multiplexing synchronization code is present in the multiplexed payload, the demultiplexer erroneously determines this bit pattern as the multiplexing synchronization code, and the MUX-PDU boundary is erroneously determined. Pseudo-synchronization (also called emulation) may be detected. When a moving image encoder generates a moving image code sequence in which a portion other than a synchronization code (VOP start code, resynchronization marker, etc.) in a moving image code sequence does not have the same bit pattern, a moving image It is possible to detect whether similar synchronization has occurred in the multiplexing layer using the image synchronization code.

  The head position of the MUX-PDU payload and the head position of the AL-PDU are combined to form a MUX-PDU. Each of the examples shown in FIG. 38 has such a configuration. Then, a moving picture synchronization code is inserted at the head of the AL-SDU (access unit). In this way, the multiplex synchronization code and the video synchronization code are arranged adjacent to each other with the multiplex header and the AL header interposed therebetween. If the multiplexing synchronization code is erroneously detected by the demultiplexer, the multiplexing header, AL header and video synchronization code adjacent to the multiplexing synchronization code are detected, but the detected multiplexing synchronization code is pseudo-synchronous. For this reason, it is assumed that there are multiplexed headers, AL headers, and moving picture synchronization codes, and completely different information is contained in the places where these are decoded. Therefore, it is determined whether or not the multiplexed header, AL header, and video synchronization code decoded by the demultiplexer are correct information. If the information is not correct, the detected multiplex synchronization code is determined to be pseudo-synchronous. .

  FIG. 48 shows a second example of the configuration of the multiplexer. In this example, the multiplexer is divided into two layers, a FlexMux layer and a TransMux layer. Further, the FlexMux layer is divided into an adaptation sublayer (AL) and a multiplexing sublayer (Mux sublayer), and the TransMux layer is divided into a protection sublayer (Protection Layer) and a transmax sublayer (TransMux Layer).

  FIG. 49 shows an example of a code string generated in the FlexMux layer. 1061A is an adaptation sublayer, and 1062A is a code string formed of a multiplexing sublayer. The adaptation sublayer has an AL header (Header) 1065A containing information such as information indicating the type of information to be multiplexed and time, and further includes a moving image code string, a voice code string, a data code string, and the like to be multiplexed. The payload (Payload) 1066A is multiplexed to generate an AL-PDU. In the multiplexing sublayer, if necessary, a FlexMux-PDU is further generated with an index (index) 1068A indicating the type and channel number of the AL-PDU and information (length) 1069A indicating the length of the AL-PDU. You.

  The FlexMux-PDU generated in the FlexMux layer is input to the TransMux layer. The configuration of the multiplexer shown in FIG. 36 may be used for the TransMux layer. In this case, the protection sublayer corresponds to the adaptation layer 1031A in FIG. 36, and the transmax sublayer corresponds to the multiplexing layer 1032A in FIG. Alternatively, the configuration of FIG. 36 may be used for the transmax sublayer, and the protection sublayer may not be used.

  Note that a configuration for multiplexing unit code strings set to an integer multiple of a predetermined length by stuffing bits and a configuration in which codewords having the same importance are collectively used as an access unit are described below. The present invention can be appropriately applied to each of the structures of the multiplexed coded sequence described in the first to third embodiments.

  When a plurality of video packets are put in one access unit as shown in FIG. 45, the boundary of the access unit and the arrangement of the resynchronization marker in the frame may be as shown in FIG. In FIG. 50, a white circle indicates a macroblock having a resynchronization marker (that is, the first macroblock of a video packet), and a gray circle indicates the position of the first macroblock in the access unit. In such an image, since a person is more important information than a background portion, it is preferable that the image has high resistance to a transmission path error. For this reason, a large number of resynchronization markers are arranged in the person portion to make the intervals between video packets small, so that recovery from a transmission path error can be achieved at an early stage, and the error resistance is strengthened. Conversely, since the background portion is not so important, the resynchronization marker may be reduced and the interval between video packets may be increased.

  Further, in an encoding method in which a frame is encoded from the upper left to the lower right macro block in the raster scan order, an error mixed in a certain macro block may spread to a lower right macro block. In particular, if an error spreads into the important area, the image quality will be greatly degraded, so the macroblock where the important area starts is the first macroblock in the access unit so that the error mixed with other access units does not affect the error. Is also good. In the example of FIG. 50, the leftmost macroblock of the person, which is an important area, is set as the first macroblock of the access unit.

  When the strength of the error protection can be switched within one access unit, the error protection may be switched according to the importance of the area in the frame. FIG. 51 shows an example in which such switching is performed. In FIG. 51, a light gray (hatched) region is a region (High QoS) in which strong error protection is performed, and is assigned to a person portion which is more important information. FIG. 52 shows an example of the configuration of an access unit corresponding to this. In the figure, a light gray (hatched) portion corresponds to the light gray macroblock in FIG. 51, and this portion performs strong error protection.

  When a video packet is divided and encoded as in Motion and Texture, as shown in FIG. 52 (b), Motion is inserted in the first half of the access unit and Texture is inserted in the second half, and is further shown in light gray in FIG. The important area may be located in the first half of each. Alternatively, as shown in FIG. 52C, the Motion and the Texture may be different access units, and the first half of each may be more strongly protected against errors. As a result, the more important Motion portion of the code string in the important area can be more strongly error-protected.

  By using the arrangement of the resynchronization markers and the configuration in the access unit as described above, it is possible to provide stronger error resilience with less overhead (redundancy). In general, overhead is increased by using a resynchronization marker or strong error protection.However, a large number of resynchronization markers are assigned to important information such as a person to strengthen error protection, and to a less important area such as a background. Reduces error protection by reducing the number of resynchronization markers, thereby providing stronger error resilience to important information with the same average overhead as compared to the case where resynchronization markers are allocated and error protected uniformly throughout. be able to.

  Furthermore, when a large number of resynchronization markers are allocated as in the case of the person portion in FIG. 51, the length of the video packet is correspondingly very short, and thus each video packet is allocated to one access unit. Then, the overhead due to the AL header, the multiplex header, the multiplex synchronization code, etc. becomes very large. In this case, the overhead is reduced when a plurality of video packets are inserted in one access unit as shown in FIG.

  The configurations and stream structures of the encoding / decoding devices of the embodiments described above can be used in appropriate combinations. The operation of each encoding / decoding device can be replaced with a processing procedure under software control and executed, and the software and encoded stream can be provided as recording media.

FIG. 1 is a block diagram showing a configuration of an encoding device used in an information transmission system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of a decoding device used in the information transmission system of the first embodiment. FIG. 2 is an exemplary view showing how a screen is divided into a plurality of layers in the information transmission system according to the first embodiment. FIG. 4 is a diagram illustrating an example of a bit string of each layer illustrated in FIG. 3. FIG. 5 is a diagram showing a configuration of another bit sequence instead of FIG. 4. FIG. 2 is a diagram showing an example of a case where a screen is configured by a single layer in the information transmission system of the first embodiment. FIG. 3 is a block diagram showing another configuration example of the encoding device used in the information transmission system of the first embodiment. FIG. 8 is a block diagram showing a configuration of a decoding device corresponding to the encoding device in FIG. 7. FIG. 4 is a diagram showing a state in a frame when refreshing is performed in the information transmission system according to the first embodiment, and an example of a bit string corresponding thereto. FIG. 4 is an exemplary view showing another example regarding the content of important information transmitted by the information transmission system of the first embodiment. FIG. 11 is a diagram showing a configuration of a decoding processing circuit corresponding to FIG. 10. FIG. 7 is a diagram showing an example in which instruction information used in the first embodiment forms a part of another header information table. FIG. 9 is a diagram for explaining a coding area in a frame used in the information transmission system according to the second embodiment of the present invention. FIG. 9 is a diagram showing an example of an image coded sequence used in the second embodiment. FIG. 15 is a view for explaining information indicating a time included in the image coded sequence of FIG. 14. FIG. 13 is a block diagram showing a configuration of a decoding device used in the second embodiment. FIG. 9 is a diagram showing an example of a VOP header and a video packet header used in the second embodiment. FIG. 13 is a block diagram showing another configuration of the decoding device used in the second embodiment. FIG. 11 is a block diagram showing the overall configuration of an image / speech encoding device used in an information transmission system according to a third embodiment of the present invention. FIG. 13 is a block diagram showing the overall configuration of an image / sound decoding device used in the third embodiment. The figure which shows an example of the moving image code string used in the 3rd embodiment. FIG. 13 is a diagram showing an example of a multiplexed code string used in the third embodiment. FIG. 14 is a diagram showing a first example of a multiplex header used in the third embodiment. FIG. 14 is a diagram showing a second example of a multiplex header used in the third embodiment. FIG. 13 is a diagram showing a second example of a multiplexed code string used in the third embodiment. FIG. 13 is a diagram showing a third example of a multiplexed code string used in the third embodiment. FIG. 9 is a diagram showing a third example of a video packet header used in the present invention. FIG. 14 is a diagram showing a fourth example of a video packet header used in the present invention. FIG. 1 is a block diagram showing a medium for recording information according to the present invention and a decoding device therefor. 30 is a flowchart showing a procedure for decoding information recorded on the medium of FIG. 29. FIG. 6 is a diagram illustrating an example in which a bit for preventing a pseudo-synchronous code is added to a code string according to the present invention. FIG. 4 is a diagram for explaining marker bits used for a code string in the present invention. FIG. 4 is a diagram showing an example of a bit string when a slice layer is used in the present invention. FIG. 14 is a diagram illustrating an example of a moving image code sequence used in a fourth embodiment of the present invention. The figure which shows the arrangement | positioning method of the synchronous code in the 4th Embodiment, and the example of a stuffing bit. The figure which shows the example of a structure of the multiplexer in 4th Embodiment. FIG. 14 is a diagram showing an example of output from an adaptation layer in the fourth embodiment. FIG. 14 is a diagram illustrating an example of output from a multiplexing layer according to the fourth embodiment. FIG. 14 is a diagram showing a first example of a method of dividing a moving image code string in an adaptation layer in the fourth embodiment. FIG. 14 is a diagram showing a second example of how to divide a moving image code string in an adaptation layer in the fourth embodiment. FIG. 15 is a diagram showing a third example of how to divide a moving image code string in an adaptation layer in the fourth embodiment. FIG. 14 is a diagram showing a fourth example of how to divide a moving image code string in an adaptation layer in the fourth embodiment. FIG. 14 is a diagram showing a fifth example of how to divide a moving image code string in an adaptation layer in the fourth embodiment. FIG. 15 is a diagram showing a sixth example of how to divide a moving image code string in an adaptation layer in the fourth embodiment. FIG. 14 is a diagram showing a seventh example of how to divide a moving image code string in an adaptation layer in the fourth embodiment. The figure showing the 8th example of how to divide a moving picture code string in an adaptation layer in the fourth embodiment. FIG. 15 is a view for explaining an example of stuffing in an adaptation layer in the fourth embodiment. FIG. 14 is a diagram showing a second example of the configuration of the multiplexer according to the fourth embodiment. FIG. 49 is a diagram illustrating an example of a code string generated in the FlexMux layer of the multiplexer having the configuration illustrated in FIG. 48 according to the fourth embodiment. FIG. 17 is a diagram for explaining another example of the arrangement of the boundary of the access unit and the resynchronization marker in the frame in the fourth embodiment. FIG. 13 is a diagram for explaining an example of switching error protection according to the importance of an area in a frame in the fourth embodiment. The figure which shows the other example of a structure of the access unit in the 4th embodiment.

Explanation of reference numerals

200, 250, 351 ... frames 201, 202, 352, 353, 354 ... slices 203, 251 ... macroblocks 361, 362, 363 ... bit strings in slices 101A ... moving picture signals 102A ... audio signals 103A ... data code strings 111A ... Moving picture coding device 112A ... Speech coding device 121A ... Moving picture coding sequence 122A ... Speech coding sequence 130A ... Multiplexer 135A ... Multiplexing code sequence 201A ... Picture start code 202A ... Picture header 203A ... Macro block coding Data 210A: Resynchronization marker 211A: Slice header 301A, 302A, 303A: Multiplexed packet 311A, 312A, 313A: Multiplexed header 321A, 322A, 323A: Multiplexed payload 310A: Multiplexed start code 351A ... multiplex code 352A ... video header information 353A ... CRC check bits 354A ... error correction code check bits 451A ... picture boundary pointer 601A, 602A ... multiplexed packet 611A, 621A ... multiplex code 612A, 615A, 622A ... PTR
1031A ... adaptation layer 1032A ... multiplex layer 1061A ... adaptation sublayer 1062A ... multiplex sublayer 1065A ... AL header 1066A ... AL payload 1068A ... index 1069A ... length information

Claims (2)

The encoded information including the image code sequence obtained by compression-encoding the image signal is divided into two or more layers, and a synchronization signal and header information necessary for decoding are added to each layer and transmitted. In an encoding device used in an information transmission system,
The information already transmitted in the upper layer or a part of the information, the same layer, as the restoration information including the information indicating the prediction mode of each image frame of the image code sequence for restoring the encoded information. Information that has already been transmitted in the or part of the information, or information for restoring the content of the information already transmitted in the upper layer or the same layer or the content of a part of the information, the encoded information Means for adding to and transmitting
Means for inserting instruction information of a predetermined bit pattern indicating that the restoration information has been added to the header information. The encoding apparatus according to claim 1, wherein the restoration information further includes information indicating a display time of each image frame of the image code string.

Images