MXPA01002481A - Multimedia information communication device and its method - Google Patents

Abstract

In a media separating unit, a data separating section separates a video packet and an audio packet, a CRC computing section carries out again the same computation carried out by the encoding side through a CRC code separating section and outputs the result of the computation to an error information adding section. The error information adding section compares the result of the computation with a CRC code added to the video packet to check whether they agree with each other. After the check, the error information adding section adds the result of the check as error information to the video packet and outputs the video packet as a new video packet to a video decoder through a buffer.

Description

EQUIPMENT AND METHOD OF MEDIA INFORMATION COMMUNICATION TECHNICAL FIELD The present invention relates generally to multi-media information communication apparatuses such as a portable visual telephone and a digital transmission receiver and methods used therein. More particularly, the present invention relates to a multi-media information communication apparatus and a method in which a stream of multiplexed coded bits carrying video data and audio data is received and demultiplexed in a media demultiplexer. , the video data being output to a video decoder and the audio data being output to an audio decoder so that the video data and the encoded audio data are decoded.

TECHNICAL BACKGROUND Japanese Patent Laid-open Publication No. 8-204584 and Japanese Patent Laid-Open Publication No. 8-65278 describe multiple media information communication apparatuses in which an error is detected in the video data taken in the bit stream • Aa.t.-, only when the video data is decoded by the video decoder in a decoding circuit in a decoding unit. A description of the method for error detection will be given in the multi-media information communication apparatus of the related art. Figure 1 shows a construction of MPEG4 video data (ISO / IEC 14496-2 standard) that are carried in a coded bit stream in a multiplexed format. The MPEG4 standard is designed to provide a reinforced tolerance to error in an encoded bit stream in order to allow image communication over channels such as a wireless channel and the Internet that are characterized by difficulty in guaranteeing communication quality. The ordinary MPEG4 video data is constructed in units known as a video pack (VP) 100 as shown in Figures 8 (a) and 8 (b). With reference to Figure 1, a video pack 100 comprises a Resync (RM) 101 marker, a video packet header 102, macro block (MB) encoded information 103 and a padding bit 104 at the end. The Resync Marker 101 comprises a unique bit pattern that does not match any of the combinations of variable length codes that occur in the bitstream. The Resync Marker 101 is provided to prevent an error from propagating when the bit stream is decoded. In the case of an error, the video decoder resumes the decoding of the video packet appropriately in a position in which the Resync Marker 101 is detected as a synchronous word. The header of the video pack 102 next to the Resync Marker 101 includes important information that is required in the decoding of the video data. The coded macro block information (MB) 103 is coded information for each macro block (MB) which constitutes a unit that is used to encode video data at the end of the coding. The fill bit 104 at the end is inserted for byte alignment of the Resync Marker 101 in the bit stream. A description of a procedure for locating an ordinary error when decoding the MPEG24 video data will be given immediately. The decoding of the video data is initiated in the Resync Marker 101 to proceed in a forward direction until an error is detected. Subsequently, the decoding of the video packet is restarted in the padding bit 104 in an opposite direction until an error is detected. According to the multi-media information communication apparatus of the related art as described above, the error correction in the video data is conducted only when the data is decoded by a decoding circuit. A disadvantage of this arrangement is that the decoding procedure that is conducted until the error is detected is wasted so that the _._ - ._..- __. - ______._ Subsequent procedure to prevent the spread of error can not be done efficiently. Accordingly, an object of the present invention is to provide a multi-media information communication apparatus in which the aforementioned problem is eliminated and in which the decoding process is not wasted in case of an error and in the which is efficiently performed a procedure to avoid the propagation of error.

BRIEF DESCRIPTION OF THE INVENTION The aforementioned objects can be achieved by a multi-media information communication apparatus for dissolving, in a data demultiplexing, a stream of encoded bits multiplexed into video data and audio data, in which the video data has adhered to them an error correction character produced at the end of an encoding according to a predetermined error correction calculation, and to decode the video data and the audio data by broadcasting the video data to a video decoder and emitting the audio data to an audio decoder, wherein the media demultiplexer comprises: demultiplexing means for dissolving the coded bit stream in the video data and the audio data; error correction character generation means to receive the video data from the > _ * _ ««. * ^ ..-_ * ___ te- demultiplexing means and subjecting them to the predetermined error correction calculation; error information addition unit for comparing a result of the calculation by means of generating error correction character with the error correction character attached to the video data, and for adhering error information based on a result of comparison to the video data, and in which the video decoder decodes the video data by reference to the error information attached to the video data by the error information addition unit. According to the invention, the multi-media demultiplexer of the multi-media information communication apparatus according to claim 1 may further comprise error correction character isolation means for receiving the video data from the demultiplexing means, isolating the error correction character from the video data, and for outputting video data detached from the error correction character to the error information addition unit. Further in accordance with the invention, the encoded bitstream of the multi-media information communication apparatus according to claim 1 or claim 2 can be formed in accordance with the standard video data compression and decoding system that is specific in the ITU-T recommendation and the ISO / IEC standard. Further in accordance with the invention, the coded bitstream of the multi-media information communication apparatus according to claim 1 or claim 2 may be an MPEG4 bitstream (ISO / IEC Standard 14496-2) that is specified in the ITU-T Recommendation and the ISO / IEC standard which describe a compression and video coding system, and each video packet in the video data 5 constituting an MPEG4 data transfer unit may have attached thereto a error correction character generated at the end of the coding according to the predetermined error correction calculation; the means of generating error correction character in the media demultiplexer can subject each video packet to the predetermined error correction calculation; and the error information addition unit in the media demultiplexer can compare a calculation result in each video packet by means of the error correction character generating means with the error correction character attached to each video packet in the video data, and adhering the error information based on a comparison result to each video packet in the video data. Further in accordance with the invention, the error information addition means of the multi-media information communication apparatus according to claim 4 can add the error information 20 at the end of each video packet in the video data. Also in accordance with the invention, the means for adding error information of the multi-media information communication apparatus according to any of claims 1 to 5. _ ^ ~ - ^^ "J ^ _? A ___« tY! Ifeá _-_. __.__ ._..., ^ ._. .. - - -.. Ti «a? Faith» _ 'j «» Ag-_ can add the error information that starts with a fixed length code comprising a single bit pattern to each video packet in the video data In accordance with the invention, the objects that are mentioned above also they can be achieved by a multi-media information communication method for dissolving a stream of encoded multiplexed bits into video data and audio data, the video data having attached thereto an error correction character produced at the end of a encoding according to a predetermined error correction calculation, and for decoding the video data and the audio data, comprising the steps of: receiving the coded bit stream and dissolving it into video data and audio data as to submit the video data to the default error correction calculation; a result of the calculation with the error correction character attached to the video data, and adding the error information based on a comparison result to the video data; and decoding the video data by referencing the error information attached to the video data. It should be appreciated that, according to the present invention, each video packet is checked by the media demultiplexer for an error, before the decoding process by the video decoder. Therefore, the decoding process of the video decoder is wasted. Error detection and error correction can be conducted efficiently so that ^ ^^^ Z ^^ f ^ High quality image reproduction is available evenly.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a diagram showing an example of construction of MPEG video data carried in a coded bit stream in a multiplexed format. Figure 2 is a block diagram of a multi-media information communication apparatus according to an embodiment of the present invention. Figure 3 is a diagram showing a construction of and a procedure performed in a media demultiplexer according to the first embodiment. Figure 4 is a diagram showing how a video packet is reviewed in the media demultiplexer according to the first embodiment. Figure 5 is a diagram showing a revised video packet in which the error information is added by an error information addition unit according to the first embodiment. Figure 6 is a diagram showing a revised video packet in which the error information is added by an error information addition unit according to a second embodiment of the present invention. Figure 7 is a diagram showing a location and configuration of error information added by an error information addition unit according to a third embodiment of the present invention; and Figures 8 (a) and (b) are diagrams showing the error information according to the first embodiment in which the construction of error information according to the third embodiment and the error information according to the second mode in which the construction of error information is used according to the third mode, respectively.

BEST WAYS TO CARRY OUT THE INVENTION For a detailed description of the invention, the best ways to carry out the invention will be discussed immediately with reference to the attached drawings.

First Mode Figure 2 is a block diagram of a multi-media information communication apparatus according to a first embodiment of the present invention. Figure 2 is derived from the ITU-T recommendation for audiovisual / multi media (H Series) (Shin Nippon ITU Society), Annex D, "Information Supplying Purpose", which shows a transmitter and a receiver of an information communication device of multiple media. With reference to Figure 2, the transmitter comprises: a video encoder 1 for compressing and encoding an image; an audio encoder 2 for compressing and encoding the audio data; a multiplexer of means 3 for multiplexing an emission of encoded video bit stream and an encoded audio bit stream emission; and volatile memories 7a and 7b. The receiver comprises: a demultiplexer of means 4 for demultiplexing the multiplexed information; a video decoder 5 for decoding the video data; an audio decoder 6 for decoding the audio data; and volatile memories 7c and 7d. The new feature of the multi-media information communication apparatus according to the present invention consists in the construction of the receiver, not the transmitter. More specifically, the novelty lies in the fact that the receiver includes the media demultiplexer 4 and the video decoder 5 as described below. An image coding system to which the present invention is applied can be any standard coding method that is specified in ITU-T recommendations, ISO / IEC standard. It is assumed that the invention according to the first embodiment described below applies to MPEG4 (ISO / IEC 14496-2 standard). _; ^ __. ^^ faith. ^^ A description of a construction of and a procedure in the demultiplexer of means according to the first embodiment will be given below. Figure 3 shows a construction of and a procedure performed in the media demultiplexer 4. As shown in Figure 3, the media demultiplexer 4 according to the first embodiment comprises a data demultiplexer 41, a character isolation unit. CRC 42, a CRC calculation unit 43 and an error information addition unit 44. With reference to Figure 3, it is shown that the media demultiplexer 4 comprises hardware members. The present invention can also be constructed so that the function of each of the units is implemented by software means, that is, by means of a program. Referring again to Figure 3, a description will be given of the operation of the media demultiplexer 4 according to the first embodiment. The coded bitstream is received by the apparatus and enters the media demultiplexer 4. The data demultiplexer 41 identifies the packet header (PH) of the packet that is carried in the encoded bit stream in a multiplexed format to determine if the packet is a video packet (VF) 100 or an audio packet (AP) 200 and to isolate the video packet (VP) 100 from the audio pack (AP) 200. The data demultiplexer 41 then issues the packet of video (VP) 100 to the character isolation unit CRC 42 and outputs the audio pack (AP) 200 to the volatile memory 7d of the audio decoder 6. In this stage, the configuration of the video package (AP) 100 is the same as a common MPEG4 video package shown in Figures 8 (a) and 8 (b). The character isolation unit CRC 42 receives the video packet (VP) 100 issued from the media demultiplexer 4 so as to detach each video packet (VP) 100 of a CRC character. The character isolation unit CRC 42 outputs the isolated CRC character to the error information addition unit 44. The character isolation unit CRC 42 outputs the isolated video packet to the CRC calculation unit 43 and the data unit. addition of error information 44. As shown in Figure 3, each of the video packages has a character at its end to check cyclic redundancy (CRC) generated at the end of the coding (transmission) by submitting specific data of the video data to a calculation that uses a predetermined polynomial. The CRC calculation unit 43 receives the video packet detached from the CRC character in the order of occurrence and submits the video packet to the same calculation used in the encoding (transmission). The CRC calculation unit 43 outputs the result of the calculation to the error information addition unit 44. The error information addition unit 44 stores the video packet and the CRC character isolated from each other by the isolation unit. CRO 42 character in internal volatile memory (not shown) or in internal memory (not shown). When the result of the calculation using the same algorithm that is used at the end of the coding (transmission) is fed from the CRC calculation unit 43, the error information addition unit 44 reads the CRC character stored in the internal volatile memory (not shown) or similar. The error information addition unit 44 compares the result of the calculation with the CRC character read in this manner as to determine whether they match. When a determination is given, the error information addition unit 44 reads the video packet detached from the CRC character and stored in the internal volatile memory or the like. The error information addition unit 44 then adds the error information indicating the determination to the video packet read in this manner. The resulting revised video pack 110 is issued to the volatile memory 7c. Each time the video packet enters from the data demultiplexer 41, the aforementioned procedure is repeated by the CRC characterization unit 42, the CRC calculation unit 43 and the error information addition unit 44 so that the revised video pack 110 is emitted to the volatile memory 7c. Assuming that the video decoder 5 is informed that the error information 105 is provided in the header of the revised video pack 110, the video decoder 5 can determine whether the error propagation occurs on a packet-by-packet basis before decoding the video data comprising the revised video pack 110 stored in the volatile memory 7c and provided with the error information 105. This is achieved by receiving the video pack 110 in the order of occurrence and making reference to the error information attached to video pack 110 each time video pack 110 is presented. Because information is given to the video decoder 5 regarding an error occurring in each revised video pack 110 before decoding the video data by referencing the error information, it is ensured that the video packet 110 is not decoded. reviewed that contains the error. Accordingly, the decoding process of the video decoder 5 is prevented from being wasted. As a result, an error is efficiently prevented from propagating. The concealment of error in the decoding using the error information according to the invention may have some variations. For example, the video package that contains an error can be replaced by a previous video package. Any suitable type of error confinement can be applied to the present invention. Figure 4 is a diagram showing how a video packet is reviewed in the media demultiplexer according to the first embodiment. As shown in Figure 3, the video packet 100 provided with a CRC C1 character is dissolved by the character isolation unit CRC 42 in the video pack 100 and in the CRC C1 character. The CRC calculation unit 43 performs a calculation that is the same as the CRC calculation at the end of the coding in which the specific data comprising the video packet 100 is subjected to the calculation using a predetermined polynomial. As a result, a result of the calculation C2 is issued. The error information addition unit 44 compares the character CRC C1 with the result of the calculation C2. When the CRC C1 character coincides with the result of the calculation C2, the error information addition unit 44 adds the error information 105, for example, the number "0", indicating that there is no error, and in the packet header of video 100, that is, before the Resync marker 01 of the video pack 100, producing the revised video pack 110 different from the video pack 100. When the CRC C1 character does not match the result of the C2 calculation, the error information addition unit 44 adds the error information 105, for example, the number "1", indicating that there is an error, in the header of the video pack 110, ie, before the marker of Resync 110. According to the first embodiment, because the CRC C1 character is isolated by the CRC character isolation unit 42 of the video package 1100, the CRC C1 character is not added to the revised video package 110. Figure 5 is a diagram showing a revised video packet in which the error information 105 is added by the error information addition unit 44 according to the first embodiment.

As shown in Figure 4, the error information 105 is added to precede the Resync marker 101 of the video pack 110. In accordance with the first embodiment of the present invention, an arrangement is established between the media demultiplexer 4 and the video decoder 5 of which the error information 105 is added in the header of the revised video pack 110. As described above, the first embodiment of the present invention is constructed such that the media demultiplexer 4 preceding the video decoder 5 checks each of the video packets 100 that constitute the video data for an error. Therefore, the video encoder 5 is given information related to an error in each of the video packets 100 that constitute the video data before decoding the video data so that the video encoder 5 can avoid Efficient way to spread an error, thus improving the tolerance to error. The first embodiment of the present invention is described as being provided with the CRC character isolation unit 42 for isolating the CRC character C1 from the video package 100. Alternatively, the CRC character isolation unit 42 can be eliminated. With this construction, the video packet 100 provided with the CRC C1 character is fed directly from the data demultiplexer 41 to the CRC calculation unit 43 and the error information addition unit 44, causing the CRC C1 character to remain contained. in the revised 110 video pack. The removal of the CRC characterization unit 42 is advantageous in that the construction is simplified and the procedure is facilitated.

Second Mode In the first embodiment, the error information addition unit 44 adds the error information to precede the Resync marker of the video packet in which an error occurs before supplying the data to the video decoder. According to the second embodiment, the error information is added to precede the Resync marker of a subsequent video pack 120. The construction of the multi-media communication apparatus according to the second embodiment is the same as the corresponding construction of the first embodiment. Therefore, only the construction of the video data comprising the revised video pack will be described immediately. Figure 6 is a diagram showing the video package 120 in which the error information is added by the error information addition unit 44 according to the second embodiment of the present invention. As shown in Figure 5, according to the second embodiment, the error information 105 is added subsequently to the fill bit 104 at the end of each video packet 120. The error information indicating an error in the packet 120 video is added to precede the Resync marker 101 of the subsequent 120 video pack. A stream comprising the Resync marker 101 in its header and the error information 105 at its end constitute the revised video packet 120. In this way, the second modality provides an advantage similar to the advantage provided by the first modality in which information related to an error in each video package 120 is available before the video decoder 5 performs a decoding procedure, avoiding therefore the propagation of error and improving the tolerance to error. In the first modality, the error information 105 is added in the header of the video pack 110. The error information addition unit 44 stores the video pack 110 in the internal volatile memory until the error information 105 is added to the video packet. 110. The second mode differs from the first mode in which the error information is added to the end of the subsequent video packet 120, ie, it is added to precede the Resync marker 101 of each video packet 120. The error information 105 is issued after the video packet 120 is issued. Accordingly, according to the second embodiment, the error information addition unit 44 is not required to store the video packet 120 in the internal volatile memory. The memory required to be adhered to the error information addition unit 44 in the first mode is not necessary, so that the construction is simplified.

In the first and second embodiments, the information 105 is described as being provided to precede the Resync marker 101 of the current video packet or the subsequent video packet. Alternatively, the error information 105 can also be provided subsequent to the Resync 101 marker.

Third Mode As described above, in the first and second modes, a provision is established between the media demultiplexer 4 and the video decoder 5 that the error information is provided to precede or follow the current video packet. The provision may also be that the error information is provided to precede or follow the previous video packet. In contrast, according to the third embodiment, the error information is provided at any desired location other than a location that precedes or follows the Resync marker 101. The construction of the multi-media communication apparatus in accordance with the third modality is the same as the construction corresponding to the first modality. Therefore, only the construction of the revised video data comprising the video package will be described below. Figure 7 is a diagram showing a location and configuration of error information added by the error information addition unit according to the third embodiment of the present invention. As shown in Figure 7, the error information 105 is not provided to precede the Resync marker 101. In contrast, the error information 105 may be provided to precede the encoded information 103 of macro block (MB). More specifically, a fixed length code comprising a unique bit pattern such as a single word 105a comprising, for example, a reserved code 000001 C5 of an MPEG4 header is followed by an error information container 105b containing the error information 105. In the case of the first and second modes, the error information 105 is provided to precede or follow the Resync marker 101, that is, it is provided to precede or follow a code comprising a bit pattern. unique that does not match any of the combinations of the variable length codes that are present in the bitstream. Since the video decoder 5 is given the information that the error information 105 is provided to precede or follow the Resync marker 101, it is ensured that the video decoder 5 recognizes the error information 105. In contrast, in the third mode, the video decoder 5 is not given any information related to where the error information 105 is provided and therefore can not locate the error information 15 unless the measure described above is used. In other words, by containing the error information 105 in the error information container 105b driven by the single word 105a, the fixed length code comprising a unique bit pattern, and giving the video encoder 5 the information that the error information 105 is conducted by the single word 105a, it is ensured that the video decoder 5 recognizes the error information 105. Accordingly, the error information 105, which is appended to the video pack 110 through the Media demultiplexer 4 is conducted by the single word 105a. The video decoder 5 can successfully detect the error information 105, by detecting the single word 105a even if the error information 105 is provided in an arbitrary location different from the immediate proximity of the Resync 101 marker of the video package and even if the location of the error information 105 is not made known to the video decoder 5. The first and second modes may also be constructed so that the single word 105a leads to the error information, as in the third embodiment. Figures 8 (a) and (b) are diagrams showing the error information according to the first embodiment, in which the construction of error information according to the third embodiment and the error information according to the second mode in which the construction of error information is used according to the third mode, respectively.

With reference to Figure 8 (a), according to the first embodiment, the error information 105 is provided to precede the Resync marker 10 of the current video pack 110. A fixed length code comprising a single bit pattern such as the single word 105a comprising, for example, a reserved code 000001 C5 of an MPEG4 header, leads to the error information container 105b which contains the error information 105 With reference to Figure 8 (b), according to the second embodiment, the error information 105 is provided to precede the Resync marker 101 of the subsequent video pack 110. A fixed length code comprising a single bit pattern such as the single word 105a comprising, for example, a reserved code 000001 C5 of an MPEG4 header, leads to the error information container 105b containing the error information 105. The resulting construction of the video data is more redundant than the corresponding construction of the video pack 110 and the video pack 120 according to the first modality alone and the second modality alone, respectively. However, unlike the first and second modalities alone, the video decoder 5 can easily detect the error information 105 of the video packet, even when the video decoder 5 is not given the information that the error information generated by the media demultiplexer 4 is provided to precede or follow the Resync marker 101 of the video packet. An advantage of the arrangements of FIGS. 8 (a) and 8 (b) over the third embodiment shown in FIG. 7 is that, as with the first and second modes alone, the error information 105 is not between the Resync marker 101 in the header of the video packet and fill bit 104 at the end thereof, the construction of the video packet remains the same as that of the related art, making it easier for the video decoder 5 to process the video packet without use any special measure. In the description given above of the first to third embodiments, the data demultiplexer 41, the character isolation unit CRC 42, the CRC calculation unit 43 and the error information addition unit 44, are described as being included in the media demultiplexer 4. Alternatively, those units may also be provided outside of the media demultiplexer 4 as long as those units are provided to precede the video decoder 5.

INDUSTRIAL APPLICABILITY It should be appreciated that, according to the present invention, each video packet is checked by the media demultiplexer for an error, before the decoding process by the video decoder. Therefore, the decoding process of the video decoder is wasted. Error detection and error correction can be conducted efficiently so that high-quality image reproduction is available evenly.

Claims (7)

NOVELTY OF THE INVENTION CLAIMS

1. - A multi-media information communication apparatus for dissolving, in a data demultiplexing, a stream of encoded bits multiplexed into video data and audio data, the video data has adhered thereto an error correction character produced at the end of an encoding according to a predetermined error correction calculation, and for decoding the video data and the audio data by outputting the video data to a video decoder and outputting the audio data to an audio decoder, wherein said demultiplexer means comprises: demultiplexing means for dissolving the stream of coded bits in video data and audio data; means for generating an error correction character to receive the video data from said demultiplexing means and subjecting them to the predetermined error correction calculation; the error information addition unit for comparing a calculation result by means of said error correction character generation means with the error correction character attached to the video data, and for adhering the error information based on a result of the comparison with the video data, and in which said video decoder decodes the video data by referring to the error information attached to the video data by said error addition unit.

2. The multi-media information communication apparatus according to claim 1, further characterized in that said multi-media demultiplexer further comprises an error correction character isolation means for receiving the video data from said demultiplexing means. , isolating the error correction character from the video data, and for outputting the video data detached from the error correction character to the error information addition unit.

3. The multi-media information communication apparatus according to claim 1, further characterized in that the coded bit stream conforms to the compression and encoding system of standard video data specified in ITU-T Recommendation and The ISO / IEC standard.

4. The multi-media information communication apparatus according to claim 1, further characterized in that: the encoded bit stream is an MPEG4 bitstream (ISO / TEC standard 14496-2) specified in ITU-T Recommendation and the ISO / IEC standard that describes a video compression and coding system, and each video packet in the video data that constitutes an MPEG4 data transfer unit have attached thereto an error correction character generated at the end. of an encoding according to the predetermined error correction calculation; said error correction character generation means in said demultiplexer means submits each video packet to the predetermined error correction calculation; and said error information addition unit in said demultiplant means compares a calculation result in each video packet by said error correction character generating means with the error correction character attached to each video packet in the data. of video, and adheres the error information based on the comparison result with each video package in the video data.

5. The multi-media information communication apparatus according to claim 4, further characterized in that said error information addition means adds the error information at the end of each video packet in the video data.

6. The multi-media information communication apparatus according to claim 1, further characterized in that said error information addition means adds the error information that starts with a fixed length code comprising a single bit pattern for each video pack in the video data.

7. A multi-media information communication method for dissolving a stream of coded multiplexed bits in video data and audio data, the video data has an error correction character produced at the end of an encoding according to a predetermined error correction calculation, and to decode the video data and the audio data, comprising the steps of: receiving the encoded bitstream and dissolving them in video data and audio data such as to submit the video data to the predetermined error correction calculation; comparing a result of the calculation with the error correction character attached to the video data, and adding the error information based on a comparison result to the video data; and decoding the video data by referring to the error information attached to the video data.