MXPA00003868A - Picture coding device and picture decoding device - Google Patents

Abstract

Multiplexing means on the side of a picture encoding device multiplexes display speed information or information expressing the absolute time, and a picture decoding device carries out processing based on the multiplexed display speed information or information expressing the absolute time. Thus, picture decoding processing may be carried out smoothly and precisely.

Description

IMAGE ENCODING DEVICE AND IMAGE DECODING DEVICE TECHNICAL FIELD The present invention relates to an image coding device / image decoding device that performs image processing.

TECHNICAL BACKGROUND Conventionally, it is always necessary on the decoding side that the analysis of the VOP header information is preceded by the analyzes of a VOP start code, a module time base and an increase in VOP time that are contained in the VOP. each VOP header, because no distinction can be made between the VOP and also analyze (information that is decimated in the case of a low-speed trigger of an image signal) and a VOP that is analyzed (information that is not decimates in the case of an image signal); therefore, there is a problem that the processing involved is difficult to manage and inadequate. To decode and synthesize the coded signals respectively that correspond to a subject, a background, a trunk and similar objects that form a pictorial image, it is necessary that each object is added with a synthesizing time signal (information that represents the absolute time) necessary to decode and synthesize the object. Without the information indicating the absolute time, an image decoding device can not synthesize the object, and therefore is unable to reconstruct the image. In summary, in the case of the generation of a pictorial image from a plurality of objects that include those that do not have information that represents absolute time, it is impossible for the prior art to combine the objects that have the required information with those that do not have this information In addition, the bit length of the module time base is increased until the next GOV header is multiplexed, which creates a problem that the bit length of the module time base continues to increase when the GOV header , which is an option, I do not know multiplex. In order to solve such problems as mentioned above, the present invention provides an image coding device and an image decoding device, whose processing accuracies improve during simple processing. Another object of the present invention is to provide an image coding device and an image decoding device that allow the generation of a pictorial image composed of a plurality of objects based on a time code. Still another object of the present invention is to avoid the generation of an unnecessary amount of information.

BRIEF DESCRIPTION OF THE INVENTION In accordance with one aspect of the present invention, an image coding device that encodes an image for each object is provided with: encoding means for encoding the image based on the predetermined display speed information; and multiplexing means for multiplexing the encoded image signal, encoded by the encoding means, with the predetermined display rate information prior to signal transmission. Due to the above, the display rate information may be sent in a multiplexed manner. According to another aspect of the present invention, the display rate information is multiplexed for each object. The above allows to multiplex the display speed information for each object. According to another aspect of the present invention, the information indicating whether the display speed of an object is fixed or variable and the information indicating the value of the display speed are multiplexed for each object. The above allows multiplexing the display speed information and the display speed value information. According to another aspect of the present invention, the display speed information contains a value representing an inherent fixed display speed and a value representing an arbitrary fixed display speed. The above allows the display velocity information decoding side to instantly specify the corresponding VOP. According to another aspect of the present invention, an image decoding device that decodes and encodes the coded bitstream of the pictorial image for each object is provided with: means of decoding display rate information to decode the information of display speed of the encoded bitstream mentioned above; and control means for controlling the reconstruction of the processed image for each object based on the display rate information encoded by the display speed information decoding means. The above allows a uniform and accurate image restoration processing with a simple structure. According to another aspect of the present invention, the display rate information is decoded for each object. The above provides increased uniformity and accuracy in the image restoration processing with a simple structure.

In accordance with another aspect of the present invention, the display rate decoding means decodes the display rate information from the coded bit stream for each object, and the control means controls the image reconstruction processing, depending on whether the display speed of the object indicated by the decoded display rate information is a fixed or variable speed. The above provides additional improved uniformity and additional increased precision in the image restoration processing with a simple structure. In accordance with another aspect of the present invention, the means of decoding information of display means decode the display rate information from the coded bit stream for each object, and the control means control the reconstruction processing. of the image by specifying each image display time based on the information representing the display speed of the object when the display speed information decoded by said display speed decoding means indicates a fixed speed, and based on in the multiplexed display time information for each image at each time in the case where the decoded display rate information indicates a variable speed. This makes the image restoration processing more uniform and more precise with a simple structure.

According to another aspect of the present invention, the display velocity information decoding means decodes the display rate information from the coded bit stream for each object, and the control means controls the reconstruction processing. of the image by specifying each image display time based on the display velocity information when the display velocity information decoded by the display velocity information decoding means indicate a fixed velocity and said fixed velocity has the value expressed by said display speed information, based on the multiplexed display time information for each image in each period when the display speed information indicates a fixed speed and the value of said fixed speed is not expressed as a particular value , and c based on the multiplexed display time information for each image and each period when the display speed identification information indicates a variable speed. This makes the image restoration processing more uniform and more precise with a simple structure. According to another aspect of the present invention, the control means are provided with: decoding time specification means that specify the time in which an object is decoded, based on the display rate information of the object decoded by the display speed information decoding means and the display speed information of the pre-set object in the decoding device, and decoding means decoding the object based on the decoding time specified by the time specification means of decoding This makes the image restoration processing more uniform and more suitable with a simple structure. According to another aspect of the present invention, an image coding device that encodes an image for each object is provided with absolute time multiplexing means by which the information representing the absolute time for each object is multiplexed into said signal of encoded image. Due to the above, the information indicating the absolute time may be the multiplexed form sent. According to another aspect with the present invention, an image decoding device that decodes and encodes the coded bitstream of an image for each object having absolute time analysis means to analyze the information indicating the absolute time for each object, and reconstructs the processed image for each object based on the information that represents the absolute time analyzed by the means of absolute time analysis. The above allows the processing of the simple and accurate image synthesis to be carried out. According to another aspect of the present invention, an image coding device that encodes an image for each object is provided with encoding time information encoding measures, such as information defining each image display time for each object, first to the time information defining the time interval between a reference time and the display time, secondly the time information defining the display time more accurately than the first time information and the corresponding image every time; the time information encoding means expresses the first time information as a bit length, and when the bit length of the first time information is greater than a predetermined set value, it repeats a bit change corresponding to the set value until the bit length becomes shorter than the set value, and at the same time it counts the number of repetitions of the bit change and encodes the number of repetitions of the bit change and a bit string that results from the bit change repeated. The above allows the reduction of the amount of encoded information that is going to be sent. According to another aspect of the present invention, an image coding device that encodes an image for each object is provided with encoding time information encoding means, such as information and defining each image display time for each object , first to the time information that defines the time interval of a reference time for the display time, secondly to the time information that defines the display time more accurately than the first time and image information which corresponds to each time; the information encoding means has the first time information that has means to keep the first time information encoded in an image of the immediately preceding time, and obtains a bit string corresponding to the difference between the first time information of an image to be encoded and the first time information of the immediately preceding encoded image that is obtained from the first time information holding means, and encodes the bit string difference as the first time information of the image to be encoded: The above ensures the reduction of the amount of encoded information that is to be sent. According to another aspect of the present invention, the image decoding device that decodes a coded bit stream of an image for each object is provided with: decoding time information decoding means, as information defining each time of image display for each object, to the first time information that defines the time interval between a reference time and the display time, the second time information that defines the display time more accurately than the first time information and the image that corresponds to each time; and decoding and synthesizing means decoding a coded picture signal inserted for each object and synthesizing the decoded picture signals. The time information decoding means decodes, as encoded data of the first time information, the number of repetitions of a bit change and a bit string resulting from the repeated bit change; and the decoding and synthesizing means, which are characterized by the decoding of the first time information by adding the bit string with a length code at a set value predetermined by the number of repetitions of the bit change, synthesize the decoded image signal based on the first and second time information decoded by the time information decoding means. The above allows the reception of an image sent with a small amount of encoded information. According to another aspect of the present invention, an image decoding device that decodes a coded bit stream of an image for each object is provided with: decoding time information decoding means, as information defining each time of image display for each object, to the first time information defining the time interval between a reference time and the display time, the second time information defining the display time more accurately than the first time information and the image that corresponds to each time; and decoding and synthesizing means decoding a coded picture signal inserted for each object and synthesizing the decoded picture signals. The time information decoding means maintains the first time information of a previously encoded image, and adds a bit string, decoded as the first time information of the image to be decoded, with the first information of time of the previously decoded image obtained from the means for holding the first time information, thus decoding the first time information of the image to be decoded; and the decoding and synthesizing means synthesize the decoded image signals based on the first and second time information decoded by the decoding and time information means. The above allows the reception of an image sent with a small amount of encoded information. According to another aspect of the present invention, a coded image signal is generated by encoding an image for each object based on the predetermined display speed information, and the coded image signal in the display speed information they are multiplexed and extracted. The above allows the reduction of the quantity and encoded information that is going to be sent. Still according to another object of the present invention, the display rate information is decoded from a coded bitstream from the image for each object, the image is encoded by controlling the reconstruction of the image encoded for each object based on the decoded display speed information. The above allows the reception of an image sent with a small amount of encoded information.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a diagram showing the structure of video data in MPEG-4; Figure 2 is a diagram showing a concrete example of VOP; Figure 3 is a block diagram illustrating a VOP encoder part according to mode 1 of the present invention; Figure 4 is a block diagram illustrating an example configuration of a header multiplexing part of the VOP coder part according to the mode 1 of the present invention; Figure 5 is a diagram for explaining a module time base and an increase in VOP time; Figure 6 is a block diagram illustrating the example of the configuration of the header multiplexing part of the VOP coder part according to the mode 1 of the present invention; Figure 7 is a block diagram showing a VOP encoder part according to mode 2 of the present invention; Figure 8 is a block diagram showing an example of the configuration of the header multiplexing part of the VOP coder part according to the mode 2 of the present invention; Figure 9 is a diagram showing an example of a bit stream; Fig. 10 is a block diagram showing an example of the configuration of the VOP header multiplexing part of a header multiplexing part according to the mode 2 of the present invention; Fig. 11 is a block diagram showing the internal configuration of a VOP encoder part according to mode 3 of the present invention; Figure 12 is a block diagram showing an example of the configuration of a header analysis part of the VOP coder part according to the mode 3 of the present invention; Figure 13 is a block diagram showing a system for synthesizing a plurality of objects according to the mode 3 of the present invention; Fig. 14 is a block diagram illustrating an example of the configuration of a header analysis part of a VOP encoder part according to the mode 4 of the present invention; Figure 15 is a block diagram illustrating an example of the configuration of the header analysis part of the VOP coder part according to the mode 4 of the present invention; Figure 16 is a block diagram showing the internal configuration of a VOP decoder part according to the mode 5 of the present invention; Figure 17 is a block diagram illustrating an example of the configuration of a header analysis part of the VOP decoder part according to the mode 5 of the present invention; Fig. 18 is a block diagram illustrating an example of the configuration of a header analysis part of a VOP of the VOP decoder part according to the mode 5 of the present invention; Fig. 19 is a block diagram showing an example of the configuration of a header analysis part of a decoder part of a VOP according to the mode 6 of the present invention; Figure 20 is a block diagram showing an example in the configuration of a header analysis part of a VOP of the decoder part VOP according to the mode 6 of the present invention; Figure 21 is a block diagram illustrating an example of the configuration of a header multiplexing part of a VOP coder part according to the mode 7 of the present invention; Fig. 22 is a block diagram illustrating an example of configuring a header multiplexing part of a VOP encoder part according to the mode 7 of the present invention; Figure 23 is a block diagram showing an example of the internal configuration of a VOP decoder part according to the mode 8 of the present invention; Fig. 24 is a block diagram showing an example of the configuration of a header analysis part of the coding part of a header analysis part of the VOP decoder part according to the mode 8 of the present invention; Fig. 25 is a block diagram illustrating a system for synthesizing a plurality of objects according to the mode 8 of the present invention; Fig. 26 is a block diagram showing an example of the configuration of the header analysis part of the VOP decoder part according to the mode 8 of the present invention; Fig. 27 is a block diagram showing an example of the internal configuration of the VOP decoder part according to the mode 8 of the present invention; Figure 28 is a block diagram showing an example of the configuration of a header multiplexing part of a VOP coder part according to the mode 9 of the present invention; Fig. 29 is a block diagram illustrating an example of the configuration of a header analysis part of a VOP decoder part according to the mode 10 of the present invention; Figure 30 is a block diagram illustrating an example of the configuration of a header multiplexing part of a VOP coder part according to the mode 11 of the present invention; and Figure 31 is a block diagram illustrating an example of a header analysis part of a VOP decoder part according to the mode 12 of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION To facilitate a better understanding of the present invention, a description will be made, with reference to the appended drawings, of the best mode for carrying out the invention.

Modality 1 In mode 1 a VOP encoder for the MPEG-4 video coding system described in ISO / IEC JTC11 SC29 / WG11 / N1796 will be described provided with constituents of this modality, ie means for encoding an image based on in the object display rate information and means for multiplexing the display rate information in a coded bit stream by adding the information for each object. The MPEG-4 system is a system that has to do with the sequence of image movement as a series of image movement objects that take arbitrary temporal and spatial forms and carries out the coding and decoding for each object of image movement . Figure 1 shows the structure of video data in MPEG-4. In the MPEG-4 system, reference is made to the image movement object that contains the time axis as a video object [video object (VO)], a VO component as a video object layer [object layer] Video (VOL)], a component of VOL as a group of video object plane [group of video object plane (GOP), and image data representing the state of GOP in each time and forms the basic unit encoding as an object plane of [video object plane (VO)] The VO corresponds, for example, to each speaker or its background in a video conference scene, the VOL forms the basic unit that has time resolutions and indirect spatial horn or background, and the VOP are the image data of said VOL in each time (corresponding to a box.) The GOP is a data structure that forms the basic unit to edit a plurality of VOLs or access random to it, such a data structure is not always necessary hoist for coding. A specific example of VOP is shown in Figure 2. Figure 2 shows two VOPs (VOP1 indicates a man and VOP2 an image on the wall). Each VOP is made up of texture data that represents the level of color graduation and the shape data represents the shape of the VOP. The texture data consists of a luminance signal of 8 bits per pixel and a color difference signal (of a sub-sampled size 1/2 of the luminance signal in the horizontal and vertical directions) and the shape data are the Same binary matrix data as the image size of the luminance signal that establishes the inside and outside of the VOP at 0 and 1, respectively. In the representation of image movement based on the VOP, a frame image is obtained by arranging a plurality of VOPs in the frame. When the image movement sequence contains only VO, each VOP is equal to the frame. In this case, there is no form data and only the texture data is encoded. A description of an image coding device of mode 1 will be described. Said description is made based on an MPEG-4 video encoder; hereinafter reference will be made to the MPEG-4 video encoder as a VOP encoder, since it carries out the coding for each VOP. Although the operation of the VOP encoder is described, for example, in ISO / IEC JTC1 / SC29 / WG11 / N1796, no description of the existing VOP encoder is made, but a description of a VOP encoder containing constituents of the VOP will be made. Figure 1. Figure 3 shows an example of the configuration of the VOP encoder in the mode 1. The reference number 110 denotes a determination part of VOP to be encoded, 111 a part of shape coding, 113 a part of motion estimation, 115 a motion compensation part, 118 a texture coding part, 122 a memory, 124 a header multiplexing part, 126 a video signal multiplexing part, 128 a subtractor, and 129 an adder. Subsequently, the operation of the encoder will be described. Based on the VOP speed 7 which is established externally or according to the coding condition, the determination part 110 of the VOP to be encoded determines the VOP to be encoded in the inserted object images, and extracts the VOP to be encoded in the form coding part 111, the movement estimation part 113 and the subtracter 128. The VOP speed 7 mentioned herein refers to a value representing the VOPs in each VOL o GOV that will be displayed per second. The speed information also mentioned herein is a keyword corresponding to the VOP speed 7, and its equivalent to what is called display rate information in the present invention.

A concrete example of the operation of the determination part 110 of the VOP to be encoded will be described. When the inserted object images are 30 / s and the VOP speed 7 is 15 / s, the determination part 110 of the VOP to be encoded qualifies those that are alternating from the VOPs contained in the inserted object images. which will be encoded, and extracts each of the other VOPs to be encoded. The VOPs specified by the determination part 110 of the VOP to be encoded as those to be encoded have their data encoded for each area with 16 by 16 pixels, which are commonly called an alpha block, have their data of Coded texture for each area with 16 by 16 pixels is called a macro block. The form coding part form 111 encodes the alpha block insert in it and extracts the information in a coded form 112 and locally decodes the information in a form 109. The encoded form information 112 is supplied to the multiplexing part of the encoder. video signal 126, while the information of locally decoded form 109 is input to the motion estimation part 113, the texture coding part 115 and the texture coding part 118. The motion estimation part 113 reads the reference data 123a of the memory 122 and carries out the coupling of the block for each macro block to obtain the movement information 114. In this case, the movement information is obtained by the coupling of blocks only for the objects contained in the macro block, in the information of locally decoded form 109. The movement compensation part 115 reads the memory 122 of the reference data 123b at the position indicated by the movement information 114 and generates a prediction image based on the information of locally decoded form 109. The prediction image 116 created in the motion estimation part 115 is inserted into the subtracter 128 and the adder 129. The subtractor 128 calculates the difference between the prediction image 116 and the macro block inserted to provide an error prediction image 117. In the texture coding part 118 the error prediction image 117 inserted in it is encoded by a predetermined method prescribed by MPEG-4 to obtain the encoded texture information 119 and the locally decoded error prediction image 120. In this case, only the objects contained in the block are encoded based on the information in a locally decoded form 109. The encoded texture information 119 is sent to the signal multiplexing part of the 126, and the locally decoded error prediction image 120 is extracted to the adder 129. The adder 129 adds the prediction image 116 and the locally decoded error prediction image 120 to create a decoded image 121, which is written to the memory 122.

In the header multiplexing part 124 the respective pieces of the header information are multiplexed, and a bit stream 125 obtained by the multiplexing of the header information is inserted into the video signal multiplexing part 126. The part video multiplexing signal 120 multiplexes the information in coded form 112, the motion information 114 and the coded texture information 119 into the bitstream 125 formed by the respective multiplexing of the header information, and extracts a stream of bits of coded VOP. Fig. 4 is a block diagram showing the configuration of the header multiplexing part shown in Fig. 3. In Fig. 4, the reference number 1 denotes a VO header multiplexing part, 2 a multiplexing part. of VOL header, 3 part of GOV header multiplexing selection, 4 a GOV header multiplexing part, 5 a VOP header multiplexing part, 6 to the GOV multiplexing information, and 7 to the VOP speed. Subsequently, the operation of this modality will be described. The VO header multiplexing part 1 creates a bit stream by multiplexing the VOP header information, and extracts the bit stream to the header multiplexing part 2.

VOL.

The VOL header multiplexing part 2 multiplexes the VOL header information into the inserted bitstream and extracts the bit stream to the GOV header multiplexing selection part 3. Based on the multiplexing information of GOV 6 which indicates whether the multiplexing of the GOV header is carried out, the GOV header multiplexing selection part 3 determines the destination of the bitstream supplied from the multiplexing part 2 of heading VOL. When the multiplexing information of GOV 6 indicates that multiplexing of the GOV header is not carried out, the bitstream extracts the VOP header multiplexing part 5, whereas when the multiplexing information of GOV 6 indicates that the multiplexing of the GOV header is carried out, the bit stream is extracted to the GOV header multiplexing part 4. Table 1 exemplifies the aforementioned VOP velocity, showing four classes of VOP velocities. When the VOP speed is 30 / s, "01" is multiplexed. When the VOP to be encoded is the same as the immediately preceding encoded VOP, the VOP information "00" is multiplexed but the VOP header information and the subsequent VOP data information are not multiplexed. When the VOP speed is variable, the VOP speed "11" is multiplexed. That is, the VOP speed 7 indicates if the VOP speed is fixed or variable, and represents the value of the speed when it is fixed. A VOP start code multiplexing part 8 in the VOP header multiplexing part 5 is extracted to a module time base multiplexing part 9 (base-time modulus) and a multiplexing part 10 of increment of VOP time (VOP-time-increment) a bitstream is obtained by multiplexing the VOP start code in the inserted bitstream. The time base of module 13 mentioned herein and the information representing the number of seconds elapsed until the VOP is displayed after a certain reference period as shown in figure 5, and the time increment of VOP 14 is the information by which the display period defined by the module time base is fine-tuned to an accuracy of 1/1000 of a second as shown similarly in Figure 5. That is, the MPEG-4 allows defining the VOP display period with an accuracy of 1/1000 of a second. A control period generation part 12 in the header multiplexing part of VOP 5 generates the time base of module 13 and the time increment of VOP 13 based on the velocity of VOP 7, and extracts the base of module time 13 to the multiplexing part 9 of the module time base and the time increment of VOP 14 to the time increment multiplexing part of VOP 10. When the VOP speed 7 represents a variable speed of VOP , the time base of module 13 and the time increment of VOP 14 are set independently of the VOP speed 7. The module time base multiplexing part 9 mentioned above multiplexes to the time base of module 13 in the bitstream provided with the VOP start code multiplexing part, and extracts the multiplexed bitstream to the multiplexing part 10 of VOP time increment. The multiplexing part 10 of the VOP time increment multiplexes to the VOP time increment 14 supplied thereto from the control time generation part 12 in the bit stream supplied from the multiplexing part 9 of the time base of module, and extracts the multiplexed bitstream to a video information header multiplexing part 11. The video information header multiplexing part 11 multiplexes a video information header into the bit stream provided therein. from the time-increment multiplexing part of VOP 10, and extracts the multiplexed bit stream to the video signal multiplexing part 126. As described above, according to mode 1, although the VOP rate is multiplexed in the GOP header, a current can be created of bits that allows the decoder side to determine whether or not it requires the decoding of the VOP, or to synthesize a plurality of objects, by analyzing the VOP start code of only each VOP header.

It is also possible to define the VOP speed for each VOL and carry out the decoding and multiplexing of the VOP speed as shown in Figure 6. In this case, the VOP speed 7 is determined for each VOL and multiplexed in the header multiplexing part of VOL 2. Based on the foregoing, the time base of module 13 and the time increment of VOP 14 are determined. As described above, mode 1 has described an example of the device encoding the image encoding the image at the base of an object and provided with encoding means for encoding the images based on the predetermined display speed information and multiplexing means for multiplexing the predetermined display speed information mentioned above in the image signals encoded by the coding means and to extract the multiplexed signals. In addition, mode 1 has described an example of multiplexing means of the type that multiplexes the aforementioned display rate information on an object-by-object basis.

Mode 2 Mode 2 refers to another example of the VOP encoder described above in mode 1. The VOP encoder of mode 2 is provided with means for encoding a speed indicator of 1-bit VOP that indicates whether the display speed information or the object display speed is a fixed or variable speed and the VOP speed that indicates the value of the object display speed, and to multiply information encoded in the bitstream. When the aforementioned VOP speed indicator indicates a variable speed, the VOP speed corresponds to "variable" in Table 1 referred previously with respect to mode 1, whereas when the VOP speed indicator indicates a fixed speed, the VOP speed corresponds to 30 / s or 15 / s in the aforementioned table 1 with respect to the mode 1. Figure 7 shows an example of the configuration of the VOP encoder according to the mode 2 of the present invention. The reference number 1000 denotes a part of header multiplexing, 1001 to the VOP speed indicator, and 1026 to the VOP speed. Although the VOP encoder according to the mode 2 differs from the VOP encoder previously described in relation to the mode 1 only in the operation configuration of the header multiplexing part 1000 corresponding to the header multiplexing part 124 thereof last, a description will be made only in this regard. Figure 8 is a block diagram showing the configuration of the header multiplexing part 1000 of the VOP encoder part according to the mode 2 of the present invention. In FIG. 8, the reference number 1002 denotes a part of the VOL header multiplexing, and 1003 a part of the VOP header multiplexing. Next, the operation will be described. The VOP header multiplexing part 1 creates a bit stream by multiplexing the VO header information, and extracts the bit stream created in this way to the multiplexing part of the VOL header. In the multiplexing part of the VOL 1002 header it multiplexes the VOL header information into the inserted bit stream, and extracts the multiplexed bitstream to the GOV header multiplexing selection part 3. In this case, the VOL speed information and the VOP speed indicator are also multiplexed. Table 3 shows examples of multiplexing the VOP speed 1026. In this case, when the VOP velocity 1026 is 2 / s, "000" is multiplexed as the VOP velocity; when the VOP speed is 5 / s, "001" is multiplexed; when the VOP speed is 25 / s, "001" is multiplexed; when the VOP speed is 30 / s, "011" is multiplexed; and in cases of other VOP speeds (for example, when the VOP speed is 10 / s), "100" is multiplexed. Incidentally, the decision is made as to whether the multiplexing of the VOP rate is made independently of the VOP indicator value described below. The multiplexing of the VOP speed can also be as exemplified in Table 4. In this case, when all the VOPs are exactly the same images in the VOP, the display is considered as a still image and "010" is multiplexed as the speed of VOP. With respect to the VOP speed indicator, "1" or "0" is multiplexed depending on whether the VOP speed indicator indicates a fixed or variable speed. Figure 9 shows an example of the bit stream provided with the header multiplexing part of VOL 1002. Based on the multiplexing information of GOV 6 it is indicated whether the GOV header is multiplexed, the multiplexing selection part of GOV header 3 determines the destination of the bitstream supplied thereto from the header multiplexing part of VOL 102. When the multiplexing information of GOV 6 does not indicate that the multiplexing of the GOV header is carried out, the bitstream is extracted to the VOP header multiplexing part 1003, whereas when the GOV 6 multiplexing information indicates the multiplexing of the GOV header, the bit stream is provided on par with the GOV header multiplexing. 4. The GOV header multiplexing part 4 multiplexes to the GOV header information in the bit stream ins and extracts the multiplexed bit stream to the VOP header multiplexing part 1003. FIG. 10 illustrates the VOP header multiplexing part 1003 in detail. In Figure 10, the reference number 1004 denotes a part of control time generation.

Later the operation of this modality will be described. The control time generation part 1004 generates a module time base and a VOP time increment based on the VOP speed 1026 when the inserted VOP speed indicator 1001 indicates a fixed speed, and based on a stopwatch contained in the VOP encoder when the inserted VOP speed indicator 1001 indicates a variable speed. The module time base and the VOP time increment created in this way are supplied to the module time base multiplexing part 9 and the VOP time increment multiplexing part 10, respectively. The multiplexing part 10 of VOP time increment multiplexes to the VOP time increment in the inserted bitstream, and extracts the multiplexed bit stream to the multiplexing part of the video information header 11. Multiplexing video information header multiplexing to the video information header in the bit stream provided thereto from the VOP time increment multiplexing part 10, and extracting the multiplexed bit stream to the signal multiplexing part of the video 126. As described above, according to mode 2, although the VOP speed indicator and the VOP speed are multiplexed in the VOL layer, a bit stream can be created that allows the user to specify their Desired VOP at a time by using the VOP speed indicator and the VOP speed on the decoder side; the foregoing allows you to determine whether or not it requires the decoding of the VOP, or to synthesize a plurality of objects, by analyzing only the VOP start code of the corresponding VOP header. Incidentally, although it is possible to distinguish between variable and fixed speeds if only the VOP speed indicator is multiplexed, the desired VOP to be decoded can be decoded. As described above, mode 2 has described an example of the image coding device that encodes the images in an object base and that is provided with: means for encoding the indicator indicating whether the object display speed is fixed or variable; multiplexing means for multiplexing the indicator in the image signal encoded by the coding means and for extracting the multiplexed signal; encoding means for encoding the images based on the predetermined display speed information; and multiplexing means for multiplexing the predetermined display speed information in the image signals encoded by the encoding means and for extracting the multiplexed signals.

Mode 3 Mode 3 will be described when applying a system wherein an image decoding device for decoding from a coded bit stream at the VOP rate mentioned previously in relation to mode 1 and for emitting it, i.e. a decoder of MPEG-4 video (hereinafter referred to as a VOP decoder) is provided for each of one of the plurality of objects and a plurality of decoded objects are synthesized to reconstruct a pictorial image. A description will first be made of the operation configuration of the image decoding device (VOP decoder) in mode 3. Although the operation of the existing VOP decoder is deciphered, for example, in ISO / IEC JTC1 / SC29? / G11 / N1796, the VOP decoder containing elements of mode 3 will be described without reference to the existing VOP decoder. The VOP decoder in mode 3 is a decoder that is capable of decoding an encoded bit stream generated by the VOP encoder previously described with reference to mode 1. Figure 11 shows an example of the internal configuration of the VOP decoder. in mode 3 of the present invention. The VOP decoder is provided with coded compressed data composed of texture and data data in a manner as previously deciphered with reference to mode 1 and shown in Figure 2, and decodes the individual pieces of data. In FIG. 11, the reference number 150 denotes the modified VOP bit stream, 151 a header analysis part, 152 a bit stream with the header information analyzed, 153 a signal analysis part of the video, 154 to coded shape data, 155 to a shape decoding part, 156 to decoded shape data, 157 to coded texture data, 158 to motion information, 159 to motion compensation part , 160 to the prediction texture data, 161 to a texture decoding part, 162 to the decoded texture data, 164 to a memory, and 165 to the reference data. With reference to Figure 11, the operation of the decoder will be described in detail. The coded VOP bit stream 150 is inserted into the header analysis part 151, where the header information is analyzed following a predetermined syntax. The bit stream having the header information analyzed in the heading analysis part 151 is supplied in the video signal analysis part 153, where the encoded texture data is analyzed in the encoded form data 154. and the motion information 158. The shape decoding part 155 decodes the encoded data inserted therein, and extracts the data in a coded form 156. The motion compensation part 159 generates the prediction texture data 160. from the reference data 165 in the memory 164 and the movement information 158 provided from the video signal analysis part 153. Based on the encoded texture data 157 and the prediction texture data 160, the decoding part 161 reconstructs the image data by the prescribed MPEG-4 method, generating the decoded texture data 162. The decoded texture data ized 162 are used for subsequent VOP decoding, and therefore are written to memory 164. Figure 12 shows the internal configuration of header analysis part 151 which is characteristic of mode 3 of the present invention. In Figure 12, the reference number 51 denotes a start code analysis part, 52 a part of the VO header analysis, 53 a part of the VOL heading analysis, 54 a part of the GOV header analysis , 58 to the VOP speed information and 55 to a part of the VOP header analysis. The heading analysis part 151 in mode 3 is characterized in that the heading analysis part of GOV 54 decodes the VOP velocity information of the VOP contained in the GOV from the bit stream and is output to the outside. A description will be made below on how to use the VOP speed information 58. The start code analysis part 51 analyzes the start code contained in the coded VOP bit stream 150 inserted therein. A stream of bits is emitted to the VOP header analysis part when the scanned start code is a VOL indicator, to the VOL header analysis part 53 when the scanned start code is VOL indicator, towards the 54 of GOV header analysis when the analyzed start code is GOV indicator, either part 55 of the VOP header analysis when the analyzed start code is an indicator of the VOP. Incidentally, upon completion of the analysis in the VOP header analysis part 55, the bit stream is output to the video signal analysis part 153. The VO header analysis part 52 analyzes the VO header information from the inserted bit stream, and outputs the resulting bitstream to the start code analysis part 51. The heading analysis part of VOL 53 analyzes the VOL header information from the inserted bit stream, and outputs the resulting bit stream to the start code analysis part 51. The header analysis part of GOV 54 analyzes the GOV header information from the inserted bit stream, and outputs the resulting bit stream to the start code analysis part 51. At this time, the velocity information of VOP contained in the GOV header information is decoded and extracted. The header analysis part of VOP 55 analyzes the VOP header information from the inserted bitstream, and outputs the resulting bitstream through the start code analysis part 51 to the signal analysis part of the VOP. video 153. By means of the VOP decoder of the above operation configuration, it is possible to extract, for each GOV, the VOP velocity information of the VOPs contained therein. Figure 3 illustrates a system that uses said information to synthesize a plurality of objects. In FIG. 13, the reference number 200 denotes a coded VOP bit stream A, 201 a coded VOP bit stream b, 202 a coded VOP bitstream c, 203a a VOP decoder part to decode the coded VOP decoder. coded VOP bit stream 200 to 200, 203b a VOP decoder part to decode the coded VOP bit stream b 201, 203c to a VOP decoder part to decode an encoded bit stream 202, 204 an image of decoded object a, 205 an image of decoded object b, 206 an image of decoded object c, 207 to speed information a of VOP, 208 to speed b of VOP, 209 to speed c of VOP, 210 to a part of composition, and 211 to a decoded pictorial image. The decoded object image mentioned herein refers to an image that is obtained by combining the decoded form data 154 and the corresponding decoded texture data 162 for each VOPs and then to the integration of said combined pieces of data for each group of VOPs (for example, GOV or VOL). The VOP bit stream encoded a200 to c202 are decoded by the VOP decoder parts a203 to c203 and correspond thereto, respectively, by which the decoded VOP images a204 to c206 are generated. At this point, the VOP decoder parts decode the velocity information a207 to c207 from corresponding VOPs, and emit them to the composition part 210. Based on the velocity information a207 to c207 of VOP, the composition part 210 determines the time of the frame in which it synthesizes the decoded VOP images in the decoded image 211, and maps them in the corresponding frame for the determined time. It is concluded, for example, that the decoded image 211 is displayed at a rate of 30 video object planes per second (which corresponds to an ordinary TV signal display rate). In addition, the following situations are concluded. The decoded VOP image a204 is displayed at a speed of 5 / s (ie, the velocity formation a207 of VOP indicates the speed of 5 / s). The decoded VOP image b205 is displayed at a speed of 10 / s (ie, the VOP velocity information indicates the speed of 10 / s). The decoded VOP image c206 is displayed at a speed of 15 / s (ie, the VOP speed information c209 indicates the speed of 15 / s). In this case, the VOP images decoded a204 to c206 are mapped in the first frame of image in each second in the decoded image 211; the decoded VOP image a204 is mapped every five image frames including the first one in every second; the decoded VOP image b205 is mapped in every 10 frames of image including the first one in every second; and the coded VOP image c206 is mapped every 15 frames of images including the first in each second. Due to the above, it is possible to visualize a pictorial image with a plurality of objects synthesized in the image frames according to their viewing speeds.

As described above, by using the VOP decoder that decodes the coded bitstream having the VOP rate information encoded in the GOV layer, a system that synthesizes a plurality of objects in a reconstructed image is You can implement with a simple structure. The VOP velocity information can also be encoded for each VOL on the side of the image coding device. In this case, it is possible, on the side of the image decoding device, to decode at the coded VOP rate for each VOL and easily synthesize a plurality of objects for each VOL as described above. Although mode 3 employs the VOP decoder as a system for synthesizing a plurality of objects, it is also possible to use only a VOP decoder for a system that decodes only one object to reconstruct an image. As described previously, according to the modality 3, the image decoding device that decodes the coded bit stream of an image on an object basis per object is provided with display velocity information decoding means for decoding the display rate information from the stream of encoded bits and control means for controlling the reconstruction of the encoded image on the object basis by object by using the visualization velocity information decoded by the display velocity information decoding means.

In mode 3, the display rate information decoding means have been described for decoding the object display rate information per object.

Mode 4 Mode 4 refers to another modification of the VOP decoder described above in mode 3. The VOP decoder according to mode 4 has a VOP specification function to be decoded based on the value of the VOP decoder. the VOP speed assumed by the decoder. Although the VOP decoder of mode 4 differs from that of mode 2 only in the operation configuration of header analysis part 151, a description will be made only in this regard. Fig. 14 is a block diagram illustrating the configuration of the header analysis part of the VOP decoder part according to mode 4, in which the VOP speed on the encoder side and the VOP speed on the decoder side they do not mate. In Fig. 14, the reference number 59 denotes the VOP selection part to be decoded, which compares a VOP rate of the GOV header analysis part 54 and an assumed VOP velocity on the side of the decoder, and outputs the VOP selection information 62. The VOP header analysis portion 55 has a counterpart 60 in addition to a header analysis part 56 of time control information and a header and information analysis part. video 57. Later the operation of this modality will be described. The selection part 59 of the VOP to be decoded emits the counterpart 60 of the VOP header analysis part VOP of VOP selection information that indicates the information about the VOP to be decoded according to the result of the comparison between the VOP speed 58 analyzed in the GOV header analysis part 54 and the VOP speed 61 assumed on the decoder side. The counterpart 60 uses the VOP selection information 62 to determine if the VOP header information is decoded proceeds to the VOP start code contained in the inserted bit stream. More specifically, when the VOP rate 58 analyzed in the GOV header analysis part 55 is 30 planes / second and the VOP speed assumed on the decoder side is 15 planes / second, the VOP selection information 62 which indicates that all other VOPs are analyzed, is extracted to counterpart 60 in part 55 of the VOP header analysis. Counterpart 60 first counts each VOP header insertion therein by a counter 60a. Subsequently, based on the count value inserted therein from the counter 60a and the VOP speed selection information 62 of the selection part 59 of the VOP to be decoded, the decision means 60b decides whether the VOP inserted you need to analyze. When it is decided that the inserted VOP needs to be analyzed, the inserted bit stream is extracted to the same time control information header analysis part 56. When it is decided that the inserted VOP does not need to be parsed, the inserted bit stream e is issued to the party to the start code analysis part 51. A concrete example will be described below. When the speed selection information is VOP 62 it is the one that needs a VOP to be analyzed for every three VOPs, the decision means 60b judges that the VOP needs to be analyzed, for which the count value of the counter 60a can be divided. between three without residue, and that the VOP does not need to be analyzed for which the count value of counter 60a is divided by three, with residue of 1 or 2. Incidentally, although the VOP decoder of mode 4 has been described due to which it is adapted for use in the case where the VOP velocity information is contained in the GOV header, the VOP velocity information may also be contained in the VOL heading as previously described with reference to the mode 2. In such a case, the heading analysis part of VOL 300 only needs to be equipped with the decoding function of the VOP velocity information. In addition, the VOP decoder of mode 4 can be used not only in systems that synthesize a plurality of objects but also in a system that decodes and reconstructs only one object. As described above, the decoder according to mode 4 has control means that are provided with: decoding time specification means for specifying the time in which an object is decoded based on the object display information decoded by the means of decoding the display speed information and the display speed information of the pre-set object in the decoding device; and decoding means for decoding the object at the decoding time specified by the decoding time specification means.

Mode 5 Mode 5 refers to another example of the VOP decoder described above in mode 4. The VOP decoder according to mode 5 is equipped with a specification function of a VOP to be decoded based on : the VOP speed indicator indicates whether the object display speed is fixed or variable; the VOP velocity information indicates the object's display speed; the externally set display control information indicates the time information externally set by a user; and a time code.

Although the VOP decoder according to the mode 5 differs, as shown in FIG. 16, from the mode 3 only in the configuration and operation of a header analysis part 1005 corresponding to the header analysis part 151 of the VOP decoder of the latter, a description will be made only in this aspect. Figure 17 is a block diagram illustrating the configuration of header analysis part 1005 of the VOP decoder part according to this embodiment. In Fig. 17, the reference number 1006 denotes a heading analysis part of VOL, 1007 a heading analysis part of GOV, 1008 a part of VOP header analysis, 1009 an externally set display control information, 1010 a VOP speed indicator, and 1011 a time code. Incidentally, the externally set display control information 1009 may be the formation indicating the absolute time or VOP selection information indicating the number of VOPs from which a VOP to be decoded is selected. Next, the operation of this modality will be described. The start code analysis part 51 analyzes the start code contained in the inserted coded VOP bit stream. The start code analysis part outputs the bit stream to the VO header analysis part 52 when the analyzed start code indicates VO, to the heading analysis part of VOL 1006 when the start code analyzed indicates VOL, to the header analysis part of GOV 1077 when the analyzed start code indicates GOV, and to the header analysis part of VOP 1008 when the analyzed start code indicates VOP. Incidentally, the bit stream is output to the video signal analysis part 153 after the end of the analysis in the header analysis part of VOP 1008. Subsequently, the header analysis part VO 52 analyzes the VOP header, the VOP speed information 58 and the VOP speed indicator 1011 contained in the inserted bitstream, and outputs the analyzed bit stream to the start code analysis part 51 and, at the same time, outputs the VOP rate 58 to the composition part 210 and the header analysis part of VOP 1008 and the VOP speed indicator 1010 to the header analysis part of VOP 1008. The heading analysis of GOV 1007 analyzes the header of GOV contained in the inserted bit stream, and outputs the analyzed bit stream to the start code analysis part 51 and, at the same time, issues the time code 1011 content in the analyzed GOV header to the header analysis part of VOP 1008. Figure 18 is a block diagram illustrating the header analysis part of VOP 1008 in detail. The reference number 1012 denotes a determination part of VOP to be decoded (1), which has a counterpart 1012a and decision means 1012b. The reference number 1013 denotes a module time base analysis part, 1014 a VOP time increment analysis part, 1015 a VOP determination part (2) to be decoded, and 1016 a selection part of the VOP determination method to be decoded. Later the operation of this modality will be described. The VOP determination method selection part to be decoded 1016 uses the VOP speed indicator 1010 to determine the destination of the inserted bit stream. When the VOP speed indicator 1010 indicates a fixed speed, the determination part (1) of VOP to be decoded 1012 is selected as the destination of the bitstream, and when the speed indicator of VOP 1010 indicates a variable speed, the time base analysis part of module 1013 is selected. A description will first be made of the case where the VOP speed indicator 1010 indicates a fixed speed. The counterpart 1012a in the determination portion (1) of VOP to be encoded 1012 increases its count value at each insertion of the bit stream in the header analysis part of VOP 1006 when the start code of the VOP is detected. VOP in the start code analysis part 51, and outputs the count value and the bitstream to the decision means 1012b. Subsequently, the decision means 1012b decides whether the VOP to be decoded needs to be decoded. The operation of the decision means 1012b will be described with respect to the first and second cases wherein the externally set display control information 1009 is provided as absolute time and wherein the externally set display control information 1009 is provided as the display information. VOP selection.

First case Based on the count value and the VOP speed information supplied to it from counterpart 1012a and timecode 1011, the decision means calculates the absolute time that the VOP candidate has for decoding. For example, in the case where the count value is 4, the VOP speed information is 2 / s and the absolute time is OhlOmOsOms, the absolute time that the VOP candidate has for the decoding is calculated as 0h10m02s0ms. This form if the calculated absolute time that the VOP candidate has for the decoding and the externally set display control information 1009 are equal to each other, it is decided that decoding is required. On the other hand, when they are not equal, the absolute time of the next VOP candidate for the decoding is calculated. The above is intended to compare the absolute time of the next VOP candidate for the absolute time decoding of the current VOP candidate for decoding, so as to ensure the decoding of the VOP of the absolute value closest to the display control information externally set 1009. The absolute time of the next VOP candidate for decoding is calculated from the already calculated absolute time of the current VOP candidate for decoding and the VOP 58 speed information. When said calculated value is less than or equal to the externally set display control information 1009, it is decided that the next VOP candidate for decoding must be decoded, and the current VOP candidate for decoding is not decoded. When the calculated value exceeds the externally set display control information 1009, any of the following methods can be selected. Decode the current VOP candidate for decoding; decoding the next VOP candidate for decoding (= not decoding the current VOP candidate for decoding); decoding the VOP of an absolute time which has a small difference between them and the externally set display control information 1009, ie, which is close to the set display control information 1009.

Second case: This is the case where the display speed is controlled on the VOP decoder side; for example, the user can determine the display speed or designate the optimal display speed according to the CPU resources. Next, the operation in this case will be described. It should be assumed that the VOP selection information indicates the need to decode one out of every three VOPs. In this case, the decision means 1012b judges that the decoding needs to be carried out when the count value supplied to them from the counterpart 1012a can be divided by 3 without a residue and that the decoding does not need to be carried out when the counter value of counterpart 1012a is divided by 3 with a residue 1 or 2. In either first and second cases, when it is judged that the VOP candidate for decoding can be decoded, the decision means emit a current of bits to the time base analysis part of module 1013, and to the start code analysis part 51 when it is judged that decoding is not necessary. The time base analysis part of module 1013 analyzes the module time base, and outputs a bitstream to the VOP time increment analysis part 1014. The time increment analysis part of VOP 1014 analyzes the VOP time increment, and outputs a bit stream to the video information header analysis part 57. The video information header analysis part 57 analyzes the video information header, and outputs a stream of bits to the start code analysis part 51. Next, a description will be made of the case where the VOP speed indicator 1010 indicates a variable speed. The time base analysis part of module 1013 analyzes the module time base, and outputs a bit stream to the time increment analysis part of VOP 1014. The time increment analysis part of VOP 1-14 analyzes the time increment of VOP, and outputs a bit stream to the of determination (2) of VOP to be decoded 1015. Based on the module time base analyzed in the time base analysis part of module 1013, the VOP time increment analyzed in the analysis portion of VOP time increment 1014 and timecode 1011, the determination part (2) of VOP to be decoded 1015 generates the absolute time of the VOP candidate for decoding, and based on the absolute time generated and the externally set display control information 1009, determines whether the VOP candidate for decoding needs to be decoded. When it is determined that the VOP needs to be decoded, a bitstream is emitted to the video information header analysis part 57, and to the start code analysis part when it is judged that decoding is not necessary. The video information header analysis part 57 analyzes the video information header, and outputs a bit stream to the start code analysis part 51.

As described above, according to the mode 5, although the bit stream with the VOP speed flag encoded and the VOP velocity information is multiplexed in the VOL layer, it is possible for the user to specify his desired VOP at a time by using the VOP speed indicator and the VOP speed; which allows it to determine if the VOP needs to be decoded, or to synthesize a plurality of objects, simply by analyzing the VOP start code contained in the corresponding VOP header information. Incidentally, when the VOPs contained in the coded VOP bit stream inserted in the VOP decoder are intra-coded, the user can specify his desired VOP at a time and cause it to be displayed. As described above, the decoder according to the mode 5 is characterized by the control means controlling the image reconstruction by specifying the image display time at each time to decode based on the speed information of the image. display when the display speed identification information decoded by the display speed information identification means indicates a fixed speed and based on the multiplexed display time information for each image at each time in the case where the information Identification of display speed indicates a variable speed.

Mode 6 Mode 6 is directed to another example of the decoder of VOP described above in mode 5. The VOP decoder according to mode 6 has a function to specify the VOP by decoding based on the VOP speed indicator that indicates whether the object display speed is fixed or viable, indicating the VOP speed the object display speed, externally established by the user of externally set display control information, and the time code. Fig. 19 is a diagram representing the header analysis part in mode 6 of the present invention. Since the VOP decoder of mode 6 differs from that of mode 5 only in the configuration and operation of the header analysis part of VOL 1006 and the header analysis part of VOP 1008, a description will be given with respect to to this only. A part of the heading analysis of VOL 1017 analyzes the heading of VOL, of information of speed of VOP and the indicator of speed of VOP contained in the input of bit flow to it, and to extract the flow of bits analyzed to the part of analysis of code of beginnings 51 and the indicator of speed of VOP 1010 to a part of analysis of heading of VOP 1018; at the same time, it extracts VOP velocity information 58 to the VOP header analysis part 1016 when the analyzed VOP velocity information indicates any set velocity value (e.g., the VOP velocity indicated by the velocity information of VOP "100" in Table 3), and VOP speed information 58 to the VOP header analysis part 1018 and the composition part 210 when the analyzed VOP velocity information indicates a particular value (e.g., the VOP speeds indicated by the VOP speed information "000", "001", "010" and "011" in Table 3). Fig. 20 is a diagram representing the header analysis part of VOP 1018 in detail. Reference number 1025 denotes a select part of the VOP determining method to be decoded and 1019 a determining part of VOP to be decoded (3), which has counterpart 1019 a, a decision part of counting value 1019 by decision means 1019 c. Reference number 1020 denotes a part of time information retention, the 1021 part of the VOP speed information calculation, and the 1022 a part of the VOP speed information retention, the 1023 part of the module time base analysis, and the 1024 part of the analysis of the rate increase. VOP time. Based on the VOP speed indicator 1010 and the VOP 58 speed information inserted thereto, the selected part of the VOP determination method can encode 1025 selects the destination of the inserted bitstream. More specifically, when the VOP speed indicator 1010 indicates a fixed velocity and the velocity information VOP 58 indicates some fixed velocity value, the determining portion of the VOP by decoding (3) is selected as the destination. When the VOP speed indicator 1010 indicates a variable speed, the operation previously described in mode 5 is performed, which will not be described again. When the VOP speed indicator 1010 indicates a fixed speed and the VOP speed information 59 a particular value, the bitstream is extracted to the VOP determining part by decoding (1) 1012. In this case, the part VOP determination by decoding (1) 1012 and the parties that follow perform the same operations as previously described in mode 5; thus, no description will be repeated. According to the above, a description will be given below of the case in which the VOP speed indicator 1010 indicates a fixed speed in the VOP speed information 58 at a fixed speed value. Counterpart 1019 a of the VOP determining portion by decoding (3) 1019 increases its count value whenever the VOP start code is detected in the start code analysis part 51 and a bit stream is fed to the VOP header analysis part 1018, and extracts the count value and the bitstream to the count value decision part 1019 b. the count value decision part 1019b extracts the bitstream and the count value to the time base analysis part of module 1023 with the count value indicating a first or a second VOP, and in the other cases , extracts the bitstream and the count value to the decision means 1019 c The time base analysis part of module 1023 analyzes the module time base and, with the count value inserted indicates the first VOP, extracts the module time base to the time information retention part 1020 and to the VOP velocity information calculation part 1021 with the count value inserted indicates the second VOP, while at the same time extracting the bit stream and the count value to the VOP time increment analysis part 1024. The time increment analysis part of VOP 1024 analyzes the VOP time increment and, when the counting pain inserted indicates the first VOP, it extracts the increment of V time OP to the time information retention part 1020 and to the VOP velocity information calculation part 1021 with the count value inserted indicates the second VOP while at the same time extracting the bit stream to the header analysis part of the VOP. video information 57. The video information header analysis part 57 analyzes the video information header and extracts the bit stream to the start code analysis part 51. The time information retention part 1020 retains the module time base of the VOP time increment inserted into it. With insertion thereto of the module time base of the VOP time increment for the second VOP, the velocity information calculation part of VOP 1021 reads the same from the time information retention part 1020 the base module time for the first VOP the VOP time increment similar to having VOP, then calculates the VOP velocity information based on which they extract VOP velocity information to the velocity information retention part of VOP 1022. will now describe a concrete example of the calculation of the VOP velocity information in the velocity information calculation part of VOP 1021 in connection with the case in which the VOP time increment is expressed with 6-bit precision. In the case where the module time base for the first VOP is "10", the VOP time increment for the first VOP is "000000" (ie, the time information about VOP is 1.0 s) , the module time base for the second VOP is "10" the VOP time increment for the second VOP is "100000" (that is, the time information about VOP is 1.5 s), the difference in information of time about both is 0.5 s. This means that the VOP to be decoded exists every 0.5 s, that is, the VOP speed is 2 / s (which corresponds to the VOP velocity information "000" in Table 3). Even when VOP speed information is not multiplexed 58, if multiplexing only the VOP speed indicator 1010 can be judged from the same as a fixed speed is indicated and, accordingly, such an operation can be carried out as described above. The VOP velocity information retention portion 1022 retains the VOP velocity information inserted therein and extracts the VOP velocity information to the part of the 210 position. The operation of the decision means 1019a will now be described. , in connection with the first and second cases in which the externally set display control information 1009 is provided as the absolute type in which the externally set display control information is provided as the VOP speed.

First case Based on the count value fed to it from the decision part shall count 1019b and the VOP velocity information provided with the VOP velocity information holding part 1022, the decision means 1019c, calculated the absolute time that the VOP candidate has to decode. With the thus calculated absolute time of the VOP candidate to decode and the externally set display control information 1009 are equal to each other, it is judged that decoding is needed. On the other hand, when they are not equal, the absolute time is calculated the next VOP candidate to decode. This is designed to make comparison between the absolute time of the next VOP candidate to decode and the absolute time of the current VOP candidate to decode, agents regulate the decoding of the VOP of the absolute value closest to the externally set display control information 1009 The absolute time of the next VOP candidate to decode is calculated from the already calculated absolute time of the current VOP candidate to decode and the VOP 58 speed information. When this calculated value is less than or equal to the display control information externally set 1009, it is decided that the next VOP candidate is decoded to decode and the current VOP candidate is not decoded to decode. When the calculated value exceeds the externally set display control information 1009, any of the following methods may be chosen. decode the current VOP candidate to decode; decode the next VOP candidate to decode (just do not decode the current VOP candidate to decode); decoding the VOP of an absolute time having a small difference between it and the externally set display control information 1009, ie, which is close to the externally set display control information 1009.

Second case When the VOP speed is derived from the externally set display control information 1009 is 2 / s and the VOP speed of the VOP 1022 speed information hold part is 4 / s, the VOP information which indicates the VOP number from which a VOP is selected to decode becomes the information that is needed to decode every 2 VOP. In this case, the decision means 1019c swears that the VOP is decoded for which the count value inserted thereto can be divided by two without residue, from part of decision value of count 19b and that has not been decode the VOP for which the count value of the count value decision part 1019b is divided by two but with residue of one. In both the first and the second case, when it is decided that the VOP candidate needs to be decoded to decode, the bit stream is extracted to the time base analysis part of module 1013, and when it is decided not to it needs to decode, the inserted bit stream is extracted to the start code analysis part 51. The time base analysis part of module 1013 analyzes the module time base and extracts the bit stream to the analysis part of increase of time of VOP 1014; the VOP time increment analysis part 1014 analyzes the VOP time increment and extracts the bit stream to the video information header analysis part 57; and the video information header analysis part 57 analyzes the video information header and extra bitstream to the start code analysis part 51.

As described above, according to the mode 6, since the bitstream is multiplexed with the coded VOP velocity indicator and the VOP velocity information on the VOL layer, and since the velocity information is calculated of VOP from the absolute times of the first and second VOP when the speed indicator VOP indicates a fixed speed, it is possible for the user to specify his desired VOP at a time by means of the use of the VOP speed indicator and the VOP speed - this allows you to determine if you need to decode the VOP involved or synthesize a plurality of objects with ease, by simply analyzing just the VOP start code contained in the corresponding VOP header information with respect to the speed of VOP sets arbitrary. Incidentally, when all the VOPs contained in the coded VOP bit stream inserted into the VOP decoder are intra-coded, the user can specify his desired VOP at a time and have it displayed. As described above, the decoder according to the mode 6 is characterized by control means controlling the reconstruction of images by specifying the time of image display at each moment to decode on the basis of the multiplexed display rate information for each image at each time in the case in which the display speed indication information decoded by the display speed information decoding means indicates a fixed speed and the fixed speed is a value not represented by the speed information of display.

Mode 7 Mode 7 is directed to another example of the VOP encoder previously described in mode 1. The VOP encoder of mode 7 has a function to add, for each VOL, the time code that defines the absolute display time of each VOP contained in the VOL involved. The time code mentioned herein is the time information exposed to the normal publication of IEC 461 for "timecode and control code for video tape recorders," which is information that defines the display time of an image in every moment that forms a cinematographic film (a picture in MPEG-2 and a VOP in MPEG-4) with a pressure of hour / minute / second. For example, in the case of carrying out video editing on a frame-by-frame basis by a commercial video adder, the addition of this information to each frame makes it possible to have access to a desired frame by simply designating the value of the frame code. weather. Since the mode VOP encoder 7 differs from the mode 1 encoder only the configuration and operation of the header multiplexing part 124, a description will be given regarding this only.

Figure 21 is a block diagram illustrating the configuration of the header multiplexing part of the VOP decoder part according to the mode 7 of the present invention; the identical parts to those of mode 1 of figure 4 are marked with the same reference numbers as in the last and no description will be repeated. Next, the operation of this modality will be described. The bit stream with the VO header information multiplexed thereon is inserted into the VO 1 header multiplexer part to the VOL 2 header multiplexer part. The VOL 2 header multiplexer part multiplexes over the bitstream inserted the VOL header information and a time code 18 which forms the basis for the time control and exposes the bitstream to the GOV 3 header multiplexing selection part. The header multiplexing selection part of the GOV 3 determines the destination of the inserted bitstream of the header multiplexing part of GOV 3 determines the destination of the inserted bit stream of the header multiplexing part of VOL 2 on the basis of the multiplexing information of GOV 6 which Indicate if the multiplexing of the GOV header is performed. When the multiplexing information of GVOV 6 indicates that no multiplexing of the GOV header is performed, the bit stream is extracted to the header multiplexing part of VOP 5, whereas when the multiplexing information of GOV 6 indicates that multiplexing of the GOV header is performed, the bitstream is extracted to the multiplexing part GOV header 4. In this case, the GOV 4 header multiplexing part multiplexes the GOV header information on the bit stream fed from the GOV 3 header multiplexing selection part and extracts the bit stream to the VOP header multiplexing part 5. The VOP header multiplexing part 5 multiplexes the VOP start code, the time control information header and the video information header on the inserted bitstream, and extracts them to the multiplexing part in video signals 126 (see Figure 3). Incidentally, the operations of the video signal multiplexing part 126 and the parts that follow are some that are described above. As described above, according to the mode 7, since the time code is multiplexed on the VOL header which is always encoded in MPEG-4, it is possible to form a bit stream that allows the creation of a composite pictorial image of a plurality of objects based on time code. In addition, in the case of making edits while encoding the bit stream encoded in accordance with mode 7 by an object video editor per commercial object, random access to a VOP at any time can be randomly accessed at an arbitrary type of objects. These objects provide increased flexibility in the synthesis of images. Incidentally, although the mode 7 encoder has been described for adding type code to each VOL, the encoder can also be configured to add the time code information for each VOP. This could be carried out with a configuration such as that shown in Figure 22 in which the time code 18 is inserted which defines the absolute display type of each VOP to the VOP header multiplexing part and be multiplexed by the same. In addition, the mode 7 has been described to involve encoding the VOP velocity information, but it is natural that the time multiplexing is independent of the VOP velocity information even when the VOP velocity information is not encoded, The same effects mentioned above are obtained. As described above, the modality image coding device 7 is provided which encodes images on an object-by-object basis with the absolute time multiplexing means by which the information representing the absolute time of each object is multiplexed. to a coded image signal.

Modality 8 Mode 8 will be described in connection with a system with a plurality of VOP decoders each of which encodes and extracts a timecode from the VOL header encoded bit stream and synthesizes a plurality of encoded objects to an image. A description will first be given of the configuration and operation of the VOP decoder in mode 8. The internal configuration of the VOP decoder in mode 8 is shown in FIG. 23. Since this encoder differs from the VOP decoder of the VOP decoder of the mode 2 only the configuration and operation of a header analysis part 302, a description will be given below with respect to this only. The header analysis part 302 has a function of decoding and extracting the time code in the VOL heading. Fig. 24 illustrates the internal configuration of the header analysis part 302. In Fig. 24, the reference number 303 denotes a heading analysis part of VOL. The start code analysis part 51 analyzes the start code contained in the inserted coded VOP 150 bit stream. The start code analysis part extracts the bit code from the VO 52 header analysis part when the analyzed start code indicates VOL, to the heading analysis part of VOL 303 when the start code analyzed indicates VO , to the heading analysis part of VOL 303 when the analyzed start code indicates VOL, to the heading analysis part of GOV 54 when the analyzed start code indicates GOV and to the heading analysis part of VOP 55 when the Start code analyzed indicates VOP. Incidentally, with the termination of the analysis in the header analysis part of VOP 55, the bit stream thereof is extracted to the video signal analysis part 153. The header analysis part of VO 52 analyzes the header VO content in the inserted bit stream and extracts the analyzed bit stream to the start code analysis part 51. The heading analysis part of VOL 303 analyzes the header information VOL in the inserted bit stream and extracts the analyzed bit stream to the analysis portion of the start code 51. In this case, the timecode 64 contained in the VOL header information is decoded and extracted. The header analysis part of GOV 54 analyzes the GOV header information in the inserted bit stream and extracts the analyzed bit stream to the start code analysis part 51: The header analysis part of VOP 55 analyzes the VOP header information in the inserted bit stream and extracts the analyzed bit stream through the start code analysis 51 to the video signal analysis part 153. With the VOP decoder of the configuration and the previous operations, it is possible to extract, for each VOL, the absolute display time of each VOP obtained in it. Figure 25 shows a system that uses this information to synthesize a plurality of objects. In FIG. 25, the reference number 400 denotes a coded VOP bit stream, 401 a coded VOP bit stream b, the 402 a coded bit stream c, the 403 a VOP decoder part for decoding the coder stream. coded VOP bits a400, 403b a VOP decoder part to decode the coded VOP bitstream b 401, 403c a VOP decoder part to decode the coded VOP bit stream c402, the 404 decoded object image c, the 405 a decoded object image b, the 406 a decoded object image c, the 407 a time code a, the 408 a time code b, the 409 a time code c, the 410 a part composition and 411 a decoded image. What is meant by decoded object image is an image obtained by combining the decoded form data 156 and the corresponding encoded texture data 162 for each of the VOPs by then integrating such combined data units for each group of the VOPs ( for example, GOV or VOL). The VOP coded bit stream a400 to the coded VOP bit stream c402 are decoded by the VOP decoding portions 403a to 403c corresponding thereto respectively, whereby the decoded VOP images a440 to c406 are generated. At that time, the VOP decoding parts decode in the time codes corresponding to 407 to c409 and extract them to the composition part 210. Based on the time codes a407 to c409, the composition part 210 determines the time of the picture of the encoded image 411 in which to synthesize the decoded VOP of each decoded object image and map them to the corresponding frame at the determined time. For example, suppose the following situations. * The composition part has a time code generation capability and determines the absolute display time of each picture frame to be synthesized. * Suppose that 01: 00: 00 is modified as the time code of the first VOP of the decoded object image a404, where 01: 00: 00 represents (hour) :( minutes) :( seconds). * Suppose that 01: 00: 10 is decoded as the time code of the first VOP of the decoded object image b405. * Suppose that 01: 00: 00 is modified as the time code of the first VOP of the decoded object image c406. * Assuming that the time code of the first picture frame of the decoded picture 411 defined in the composition part 410 is 01: 00: 00, the decoded object picture a404 is mapped to the first decoded picture frame 411, the decoded object image v405 10 seconds after the first frame of the decoded image 411 and the decoded object image c406 is mapped one minute after the first frame of the decoded image 411; thus, a visualization object of the decoded objects can be realized. By this, it is possible to display a pictorial image with a plurality of synthesized video objects of the image frames in correspondence with the absolute reference times.

As described above, by using the VOP decoder decoding the decoded bit group having the decoded timecode in the GOV layer, a scheme that synthesizes a plurality of objects to a reconstructed image can be carried out with a simple structure. . The time code for each VOL can also be encoded on the image coding device side as shown in Figure 26. In this case, it is possible, on the decoding and image device side, to code the code of the decoding device. encoded time for each VOL and synthesize a plurality of objects for each VOL as described above. It is also possible to configure such a VOP decoder as shown in FIG. 27 in which a bit stream encoded with the VOP rate multiplexed thereon is inserted into the VOL header, along with the time code. In this case, since the absolute dissolution time of the first VOP of the VOL is determined by the time code and then the absolute display time of each VOP can be easily infected, from the VOP velocity information, it is feasible configure a system that synthesizes a plurality of objects more easily. While the mode 8 employs the VOP decoder as a system for synthesizing a plurality of objects, it is also possible to use only a VOP decoder in a system that decodes only one object to reconstruct an image. As described above, according to the embodiment 8, the image decoding device that decodes the decoded bit time of an image on an object-by-object basis is provided with absolute analysis means to analyze, for each object , and formation that indicates the absolute type for the same and types to reconstruct the image processed on the basis of object by object through the use of the information that indicates the absolute time analyzed by the means of absolute time analysis.

Modality 9 Mode 9 will be described in connection with an improved module time base coding method and a VOP encoder for it which is used to represent the module time base (corresponding to the first time information). VOP time increment (corresponding to the second time information) that is now used in MPEG-4. A description will first be given of the method for representing the time base of module 20 in MPEG-4. As described previously in mode 1, the value of the module time base is this information that indicates how many seconds will pass until the VOP involved is displayed after a certain reference time as shown in figure 5 and the information expresses the number of seconds in terms of the bit number of the value "1". The end of the data is clearly seen by adding the value "0". That is, when the visualization is provided after 5 seconds, the information becomes "111110". With this method, with the reference time does not change at all, the amount of information of the module time base that increases infinitely. Currently, MPEG-4 defines the reference time by the time code that is multiplexed over the GOV header, but since the GOV header is an option, it is not always necessary to encode the GOV header in MPEG-4 transcripts . That is, there is always a fear that the value of the module time base will become unlimitedly longer unless the GOV header appears. The mode 9 takes effect an encoder which avoids such problem in the coding of the data of the module time base. The mode 9 requires modifying only the configuration and the operation that the header multiplexing part 124 of the VOP coders described so far, will focus on a description on this part only. Figure 28 illustrates the internal configuration of the header multiplexing part 124 in the mode 9 of the present invention. IN reference number 500 denotes a part of VOP header multiplexing, 19 a part of the bit length calculation, 20 a modulo type base, 21 a modulo time base changed, 22 a bit of information indicating a repetition count and 501 a multiplexing part of module time base. Next, the operation of this modality will be described. The bit stream with VO header information multiplexed thereon is inserted into the header multiplexing part of V01 to the header multiplexing part of VOL2. The header multiplexing part of VOL2 multiplexes the VOL header information on the inserted bitstream and extracts the multiplexed bitstream to the multiplexing and header selection part of GOV3. In the header multiplexing selection part of GOV3 it determines the bitstream style of the multiplexing part of the header of VOL2 on the basis of multiplexing of GOV6 which indicates whether the multiplexing of the GOV header is performed. When the multiplexing information of GOV6 indicates that no multiplexing of the GOV header is performed, the bitstream is extracted to the header multiplexing part of VOP5, whereas when the multiplexing information of GOV6 indicates that multiplexing is performed of the GOV header, the bitstream is extracted to the multiplexing part of GOV4. In this case, the GOV4 header multiplexing part multiplexes the GOV header information on the bitstream from the multiplexing bending part of the GOV3 header and extracts the multiplexed bitstream to the multiplexing part of the GOV4 header. VOP5 header. The VOP8 start code multiplexing part in the header multiplexing part of VOP500 multiplexes the VOP start code on the inserted bitstream and extracts the multiplexed bitstream to the time base multiplexing part of module 501 The calculation part of the bit unit 9 in the header multiplexing part of VOP500 compares the bit length of the module time base 20 and a positive minimum value present; when the bit length of the modulo time base 20 is longer than the minimum value, the time base of modulo 20 is changed to the left repeatedly by the length of the minimum value until the length of the time base of the module becomes shorter than the minimum value, and the time base of module 21, which is the requesting bit string, and the information bit 22, which indicates the repetition count of change, are issued. The information bit 22 indicating the change repetition count may be provided as a binary number expressing the change repeat count by a predetermined bit number, or as a variable bit length expressing the change repeat count for a variable length code. A concrete example of the operation in the bit length calculation part 19 will be described below. With the minimum value before said set in 4, if the time base of module 20 is "1111111110", the repetition count of change is two and the modulo 21 time base changed is "10". If it is expressed as a fixed length of two bits, it outputs information 22 indicating the repetition count of change is "10". The time-base multiplexing part of module 501 in the header multiplexing part of VOP500 multiplexes on the bit stream from the VOP8 start code multiplexing part the modulo 21 time base changed emits information 22 which indicates the change repeat count and extracts the multiplexed bitstream to the time increment multiplexing part of VOP10. The time-increment multiplexing part of VOP10 multiplexes the time increment of VOP over the bitstream from the time-based multiplexing part of module 10, extracts the flow of the multiplexed bit to the header multiplexing part of the module. video information 11. The video information header multiplexing part 11 multiplexes the video information header about the bit stream from the time-increment multiplexing part of VOP10 and extracts the multiplexed bitstream to the part of video signal multiplexing 26. As described above, according to mode 9, the module time base is expressed by two kinds of information bits (the time base of the changed module and the information bit that indicates the repetition count change) and these two kinds of information bits are multiplexed instead of multiplexing the time base of the express module as currently prescribed in MPEG-4; therefore, it is possible to suppress the amount of information generated compared to the method according to MPEG-4. As described above, the image rating device of the mode 9 which encodes images on the basis of object by object is provided with encoding means of information in time that it encodes, as information that defines the time of visualization of an image in a time on the basis of object by object, the first time information that defines the time interval between the reference time and the display time and the second information that defines the display time with precision higher than the time defined by the first time information and the corresponding image at each time; the time information encoding means expresses a first time information per conversion to a bit length and when the bit length of the first time information is longer than a predetermined set value, a bit change corresponding to value is repeated established until the closing of bits is made shorter than the set value and, at the same time, the number of bit changes is counted, and the repetition count of change is then coded and the bit string obtained by the repetition of the bit change.

Modality 10 Mode 10 is directed to a VOP decoder that decodes multiplexed module time base information on the coded bit stream in the module time base multiplexing part described above in mode 9 and uses the information decoded and the VOP time increment to define the display time of each VOP. Since the mode differs from the VOP decoders described so far only one configuration and the operation of the header analysis part 151, a description will be seen with respect to this only. Figure 29 illustrates the internal configuration of header analysis part 151 in mode 10 of the present invention. The reference number 502 denotes a part of the VOP header analysis, 65 a part of the module time base analysis, 66 a part of the VOP time increment analysis, 67 a part of the time base calculation of modulo changed and 70 a bit of information indicating a repetition count of change. Next, the operation of this modality will be described. A start code analysis part 51 analyzes the start code contained in an encoded disk stream having multiplexed on it the time base of module 69 changed and inserted and the information bit 70 indicating the repetition count of change and extract the bitstream 152 to the VO 52 header analysis part with the scanned start code is contained in the VO header, to the header analysis part of VOL 53 with the scanned start code being contained in the VOL heading, to the header analysis part of GOV 54 when the scanned start code is contained in the GOV header, to the header analysis part of VOP 55 when the scanned start code is contained in the VOP header, and the video signal analysis part 153 (see FIG. 11) when the scanned start code is contained in the data information d e VOP. The operations of the video signal analysis part and the parts that follow are the same as those described so far. The time base analysis part of module 65 in the header analysis part of VOP 502 analyzes the time base of modulo 69 changed and the information bit 70 indicating the repetition count of change contained in the bit stream fed from the start code analysis part 51 and extrudes the modulo 69 time base changed and the information bit 70 indicating the change repetition count to the module 67 time base calculation part and the flow of bits to the time increment analysis part of VOP 66. The time base calculation part of module 67 calculates the module time base of the modulo 69 time base changed and the information bit 70 that indicates the repetition count change and extracts it from the composition part 210. More specifically, the value of the module time base is stored by inverting the procedure previously described with reference to the modality 9. In the or in which the preset positive minimum value (the decoder side is also required to set exactly the same value as the minimum value described with respect to the mode 9 encoder) and the modulo 69 time base changed "10" and the information bit 70 indicating the change repeat count is "1111111110 with" 11111111"added to the high order bit of" 10"is the restored value of the module time base. The restored value thus obtained is used from the module time base to define the VOP display time involved, together with the VOP type increase information. The time increment analysis part of VOP 66 analyzes the increase in VOP time contained in the flow of bit fed from the time-based analysis part of module 65 and extracts the analyzed bit stream to the video information header analysis part 57. The video information header analysis part 57 analyzes the header of video information contained in the bitstream fed from the time increment analysis part of VOP 66 and extracts the analyzed bit stream to the video signal analysis part 153. As described above, the mode 10 decoder is configured to calculate the module time base of the two information bit classes (the modulo time base changed and the information indicating the change repetition count); it is therefore possible to analyze the bitstream described later in the mode 12 having less amount of information generated than that by the coded representation described in MPEG-4.

As described above, the modality image display device 10 that decodes a bit stream with encoded images on an object-by-object basis is provided with: decoding time information decoding means, such as the defining information the time of visualization of an image of each time on the basis of object by object, and the first time information that defines the time interval between the reference time and the time of visualization and the second information that defines the time of visualization with higher precision than the time defined by the first information and the corresponding image at each time; and decoding and synthesizing means for decoding the encoded image signal inserted on the object-by-object basis and synthesizing those decoded image signals. The time information decoding means expresses a first time information per conversion to a bit length and when a bit length decodes, as encoded data of the first time information, a bit street derived from the repetition count of change of bit and of repeated bit change and decodes the first information by adding a bit string with code of a length of the set value predetermined by the bit change repetition count and the decoding and synthesizing means synthesize the decoded picture signal over the basis of the first and second time information decoded by the time information decoding means.

Modality 11 Mode 11 will be described in connection with another improved time base coding method and a VOP coder for it which it uses to represent the module time base and the VOP time increment that are now used in MPEG-4 Since the mode 11 differs from the VOP coders described so far only in the configuration and operation of the header multiplexing part 124, a description will be given regarding this only. Figure 30 illustrates the internal configuration of a header multiplexing part 124 in mode 11. Reference number 503 denotes a VOP header multiplexing part, 23 a module-type base retention portion, 24 a base generation part of the difference module type, the one part multiplexing module type base and the 26 a difference module time base. The start code multiplexing part VOP 8 in the header multiplexing part of VOP 503 multiplexes the start code of VOP over the entered bitstream and extracts the multiplexed bitstream to the time base multiplexing part of the VOP. difference module 25. The module time base retention means 23 in the VOP header multiplexing part 503 retains the value of the VOP module time base encoded immediately prior to and then extracted from them the immediately preceding coded VOP module time base, the modulo time base of the VOP is written to be coded in the module time base retention part. The time modulation time base generation part 24 in the VOP header multiplexing part 503 calculates a bit string of the difference between the module time base in the immediately preceding encoded VOP inserted thereto from the module time base retention part 23 and the module type base of the VOP to be decoded, then calculates the difference time module base 26 based on the number of bits "1" contained in the bit string of calculated difference and extracts it from the time base multiplexing part of difference module 25. Now, a concrete example of the generation of the difference module time base will be described. In the case in which the module time base of the immediately pre-coded VOP is "11110" (decimal number: 30) and the module time base of the VOP to be coded is "111110" (decimal number: 62), the string of difference bits becomes "100000" (decimal number: 32). Then, the number of bits "1" contained in the difference bit string thus calculated "100000" is one. In the case of calculating the difference module time base by such a conversion table as in table 2, the time base in the difference module corresponding to a "1" bit is "10" and, therefore, is extracted "10" as the time base of difference module. Table 2 is an example of the conversion table and you can also define other conversion tables. It is also impossible to obtain time base of difference module by simply making a comparison of the bit durations only. For example, in the previous example, the bit length of the VOP module time base encoded immediately prior is 5 and the base time bit of module time of the VOP to be encoded is 6, therefore, you get a value of 1 as the difference. Using this value as a substitute for the "number of bits" 1"content in the string of difference bits" in Table 2, the difference module time can be expressed. The time-division multiplexing part of difference module 25 in the VOP header multiplexing part 503 multiplexes the time base of difference module 26 over the inserted bitstream and extracts the bitstream multiplexing to the part of VOP time increment multiplexing 10. In the time increment multiplexing part of VOP 10 in the VOP header multiplexing part 503 multiplexes the time increment of VOP over the bit stream fed from the multiplexing part of the VOP. time difference module module 25 and extracts the multiplexed bit stream to the video information header multiplexing part 11. As described above, the encoder according to the mode 11 is adapted to express module time base as the time base of difference module and multiplexes the time base of difference module instead of coding the base of time or module in the manner previously prescribed in MPEG-4; therefore, the amount of information generated can be made smaller than in the case of using the method prescribed in MPGE-4. As described above, the image coding device of the mode 11 which encodes images on the basis of object by object is provided by encoding means of time information that they encode, as information that defines the display time of an image in each time on the basis of object by object, the first time information that defines the time interval between difference bits and the display time and the second information that defines the type of display of an image at each time based on object by object, the first time information defining the time interval between the reference time and the display time and the second information defining the display time with a higher position than with the first time and image information corresponding to each time; the time information encoding means have the means of retaining the first time information to retain the first time information encoded for the image in the immediately preceding time and calculates a bit string of the difference between the first time information of the image to be encoded and the first time information of the image in the immediate preceding time provided with the first time information retention means and encodes the difference bit string as the first time information of the image to be encoded.

Mode 12 Mode 12 is directed to a VOP encoder which restores the value of the module time base of the VOP involved from the information about the multiplexed difference module time base over the bit stream encoded in the split time base multiplexing part 25 described above in mode 11 and uses the restored modulo time base value to define the display time of each VOP. Since the mode 12 differs from the VOP described so far only in the configuration and operation of the heading analysis part 151, a description will be given regarding this only. Figure 31 illustrates the internal configuration of header analysis part 151 in mode 12 of the present invention. The reference number 504 denotes a part of the VOP header analysis, from 71 a part of the difference time base analysis, the 72 a separate generation of the module time base, the 73 a separate increment analysis of VOP time, 74 a module time base retention part and 75 a difference module time base. The time base analysis part of difference module 71 in the header analysis part of VOP 504 analyzes the difference time base 75 contained in a bit stream fed from the start code analysis part 51 and is in the time base time base of difference module 75 analyzed to the time base generation part of module 72 and the analyzed bit stream to the time increment analysis part of VOP 73. The generation part module time base 72 in the header analysis part of VOP 504 calculates the number of bits "1" contained in the bit string of the difference between the module time base and the previously analyzed VOP immediately the time base of the VOP module to be analyzed, based on the time base of the difference module 75 analyzed on the basis of the conversion table shown in Table 3, then generates a module time base from n calculated number of bits "1" and the module time base of the analyzed VOP immediately immediately available from the time base retention portion of module 74 and extracts the module time base thus generated to the retention portion of the module. module time base 74. A concrete example of the generation of the module time base will be described.

Assume that the timebase of difference module analyzed is "10" and that the module time base analyzed immediately prior and retained in the module time base retention part is "11110". In the case of calculating from the conversion table shown in Table 3, the number of bits "1" contained in the bit string of the difference between the time base of the VOP module analyzed immediately beforehand and the base of module time the VOP will be to analyze, it is known that the number of bits "1" contained in the difference bit stream corresponding to the time base of difference module "10" is one. Then, a bit "1" is added to the most significant bit of the module time base "11110" of the VOP analyzed immediately prior to obtaining a module time base. The conversion table in Table 2 is an example and other conversion tables can be defined and used. The restored value of the modulo timebase is used to define the dissociation time of the VOP involved, together with the VOP time increment information. In addition, the "bit numbers" 1"contained in the bit string of the difference between the module time base and the previously analyzed VOP immediately and the module time base of the VOP to be analyzed" can also be a encoded bit stream as the "difference value" between the bit length of the module time base of the VOP analyzed immediately beforehand and the bit length of the module time base of the VOP to be analyzed "and in this case it is necessary to change only the interpretation of such a conversion table as the frame 2. The time base retention portion of module 74 in the header analysis part of VOP 504 retains the module time base of the previously analyzed VOP immediately after which the base of module time of the coded VOP immediately preceding, the module time base of the VOP is inserted by encoding the module time base retention part. The time increment analysis part of VOP 73 in the header analysis part of VOP 504 analyzes the time increment of VOP contained in the fed bit stream of the time base analysis part of difference module 71 and extracts the analyzed bitstream from the 57 video information header analysis part. As described above, the mode encoder 12 is adapted to calculate the module time base of the difference time module basis by a small amount of information; it is therefore possible to analyze the bitstream previously described in the mode 8 having a smaller amount of information than that by the encoded representation described in MPEG-4. As described above, the modality image coding device 12 that decodes the bitstream with images encoded on an object-by-object basis is provided with: Time information decoding means that decodes the first time information that defines the time interval between the reference time and the display time and the second information that defines the type of display with a pressure higher than the time defined by the first time information, as information that defines the time of visualization of an image at each time in a series of images and the corresponding image at each time; and means of encoding and synthesizing to decode the encoded image signal inserted on an object-by-object basis and synthesizing these decoded image signals. The time-forming decoding means retains the first time information of the immediately decoded image, then adds the first time information of the decoded image immediately available from the first time information retaining means to a chain of time. bits decoded as the first time information of the image to be decoded, thus decoding the first time information of the image to be decoded; and the decoding and synthesizing means synthesizes the decoded image signal on the basis of the first and second time information decoded by the time information decoding means.

Modality 13 While modes 1 to 12 have been described above, the image coding device multiplexes the display rate information on the encoded image signal and that the image coding device multiplexes the absolute time information on the signal of coded image, it is so impossible to carry out an image coding device that multiplexes both the display rate information and the absolute time information on the coded image signal. This can be done by a parallel or serial arrangement of multiplexing information of display speed information and means of multiplexing absolute time information in the respective image coding device described in each of the modes 1 to 12. The same is valid for the side of the image decoding device. To simply express it, previously prescribed in the modes 1 to 12 that the image decoding device decodes the display speed information and uses this decoded display rate information to reconstruct processed images on the basis of object by object and that the device the image decoding decodes the absolute time information and uses the decoded absolute time information to reconstruct processed images on the basis of object by object; however, it is so impossible to effect an image decoding device that reconstructs the processed images for each object on the basis of the display speed information and the absolute time information.

This can be done by a parallel or serial arrangement of a display speed information decoding part and an absolute time information decoding part in the respective image decoding device described in each of the modes 1 to 12, in such a way that processed images are reconstructed for each object based on decoded information in each decoding part. With the above configuration, it can be performed in the restoration and synthesis of images more easily and more accurately.

Modality 14 While modes 1 to 13 have been described above, the image coding device encodes and multiplexes the display rate information on the encoded image signal and that the image coding device multiplexes the first time information, the second time and image information is so impossible that an image coding device that encodes and multiplexes the display speed information, the first time information, the second time information and the image is effective. This can be done by a parallel or serial arrangement of the display rate information multiplexing means and the first and second time information and the image multiplexing means in the respective image coding device described in each one of the modes 1 to 13. The same is valid for the side of the image coding device. To express it briefly, it has been described above in the modes 1 to 13 the image decoding device decodes the display speed information and, based on the decoded display speed information, reconstructs processed images for each object and that the device of image decoding decodes the first time information, the second time information and the image and, based on the first decoded time information, the second time information and the image, reconstructs the image; however, it is so impossible to effect an image decoding device that reconstructs images on the basis of the display rate information and the first and second decoded time information. This can be done by a parallel or serial arrangement of the display speed information decoding part and the time information decoding part in the respective image decoding device described in each of the modes 1 to 13 , so that processed images are reconstructed for each object based on the decoded information in each decoding part (means).

With the above configuration, the restoration of images can be done more easily and more accurately with a small amount of encoded information sent.

Modality 15 While it has been described above in embodiments 1 to 14 that the image coding device multiplexes the absolute time information and the encoded image signal and that the image coding device encodes and multiplexes the first time information, the second time information and the image, it is also possible to carry out an image coding device which encodes and multiplexes the absolute time information, the first and the second time information and the image. This can also be achieved by a parallel or serial arrangement of the absolute time multiplexing means and the first and second time information and the coding and multiplexing means of images in the respective image coding device described in each case. one of the modes 1 to 14. The same is valid for the side of the image decoding device. To simply express it, it has been described above in embodiments 1 to 14 that the image decoding devices decode an absolute time information and, based on the decoded absolute time information, reconstruct processed images for each object and that the image decoding decodes the first time information, the second time information and the image and reconstructs the image, based on the first decoded time information, the second time information and the image; however, it is also It is possible to carry out an image decoding device that reconstructs images based on the absolute time information and the first and second decoded time information. This can also be achieved by a parallel or serial arrangement of the absolute time information decoding part and the time information decoding part in the respective image decoding device described in each of the modes 1 to 13, so that the processed images for each object are reconstructed based on the decoded information in each decoding part (means). With the above configuration, image restoration can be achieved more easily and more accurately with a small amount of encoded information sent.

INDUSTRIAL APPLICABILITY As described above according to the present invention, the image decoding device analyzes the multiplexed display speed information in the image coding device and performs the decoding based on the analyzed display rate information, allowing thus the phase of reconstruction of images with a simple structure. Furthermore, the image decoding device decodes the multiplexed absolute time information in the image decoding device and performs the decoding based on the analyzed absolute time information, thus allowing the reconstruction of images with ease and high precision. In addition, the image decoding device decodes the first and second coded time information in the image coding device and decodes the inserted image signal based on the first and second decoded time information thus allowing the reception of the signal of image with a small amount of information sent.

TABLE 1 TABLE 2 • 1"CONTINUED FOR n BITS TABLE 3 TABLE 4

Claims (19)

NOVELTY OF THE INVENTION CLAIMS

1. - An image coding device that encodes images for each object, characterized by: encoding means that encode an image based on the predetermined display speed information; and multiplexing means multiplexing said predetermined display speed information on the encoded image signal, encoded by said encoding means and extracting the multiplexed signal.

2. The image coding device according to claim 1, further characterized in that the multiplexing means multiplexes display rate information for each object.

3. The image coding device according to claim 1, further characterized in that the multiplexing means multiplexes, as said display speed information, information indicating whether the object display speed is fixed or variable speed.

4. The image coding device according to claim 1, further characterized in that the multiplexing means multiplexes, as said display rate information, a value indicating the object display speed.

5. - The image coding device according to claim 4, further characterized in that the display speed information includes a value indicating an inherent fixed display speed and a value indicating an arbitrary fixed display speed.

6.- A decoding device of mage that decodes a coded bit stream that is formed by coding an image for each object, further characterized by: decoding means of display speed information that decode velocity information of said flow of coded bits; and control means which, based on the display speed information decoded by said display speed information decoding means, control the reconstruction of the processed image for each object.

7. The image decoding device according to claim 6, further characterized in that the display speed information decoding means decodes the display speed information for each object.

8. The image decoding device according to claim 6, further characterized in that: the display rate information decoding means decode the coded bit stream display speed information (for each object); and the control means controls the image reconstruction, depending on whether the object display speed indicated by the decoded display rate information is fixed or variable speed.

9. The image decoding device according to claim 8, further characterized in that: the display speed information decoding means decode the display rate information of the coded bitstream for each object; and the control means controls the reconstruction by specifying the display time of an object at each time based on the object display speed value in the decoded display speed information in the case in which the display speed information decoded by said display speed information decoding means indicates the fixed speed and on the basis of multiplexed display time information for each picture at each time in the case in which the decoded display rate information indicates the variable speed .

10. The mage decoding device according to claim 6, further characterized in that: the display speed information decoding means decode the coded bit stream display rate information for each object; and the control means controls the reconstruction by specifying the display time of an object at each time based on the display velocity information in the case in which the display velocity information decoded by said decoding information means. display speed indicates the fixed speed and said fixed speed is a value indicated by said display speed information, based on the display time information multiplexed by each image at each time in the case in which the speed information display indicates the fixed speed and said fixed speed is not indicated as a specific value and on the basis of the multiplexed display time information for each image at each time in the case in which the display speed information indicates the speed variable.

11. The image decoding device according to claim 6 or 7, further characterized in that the control means are provided with: decoding type specification means that specify the decoding time of the object based on the information of decoded object display speed, decoded by the display speed information decoding means and the pre-set object display speed information in the decoding device; and decoding means performing the object decoding on the basis of the object decoding time obtained by said decoding time specification means.

12. An image encoding device that encodes an image for each object, further characterized by absolute time multiplexing means that multiplex, for each object, information indicating absolute time for said object on said encoded image signal.

13. - An image decoding device that decodes a coded bit stream that is formed by coding an image for each object, further characterized by: absolute time analysis means that analyze, for each object, information indicating absolute time for said object; and control means controlling the reconstruction of a processed image for each object based on the absolute time information analyzed by said absolute time analysis means.

14. An image encoding device that encodes an image for each object, further characterized by time information encoding means for encoding the first time information defining the time interval between a reference time and a viewing time and the second time information defining the display time with precision higher than the time defined by said first time information, as information defining the time of image display at each time for each object, and an image corresponding to each time, and wherein when the bit length of said first time information is longer than a predetermined set value, said information encoding means repeats a bit change of said set value until the bit length becomes more short that said set value and count the bit change number and code that change number ios of bits and a bit string obtained by the repeated bit changes.

15. - An image encoding device that encodes an image for each object, further characterized by time information encoding means for encoding the first time information defining the time interval between a reference time and a display time and the second time information that defines the display time with higher precision than the time defined by said first time information, as information that defines the time of visualization of an image at each time for each object, and an image corresponding to each time , and wherein said means of time information encoding have first time information retention means for retaining the first time information of an encoded image in an immediately preceding time and calculates a bit string of the difference between the first information of time of an image to be encoded and the first information of The time of the image in the immediately preceding time available from said first time information retention means and then encodes said difference bit string as the first time information of the image to be encoded.

16. An image decoding device that decodes a bit stream that is formed by coding an image for each object, further characterized by: means of decoding time information to decode the first time information defining the time interval between a reference time and a display time and the second time information defining the display time with higher precision than the time defined by said first time information, information that defines the display time of an image at each time for each object, and a map corresponding to each time; and decoding and synthesizing means for decoding inserted encoded picture signals for each object and synthesizing these decoded picture signals, and wherein said time information decoding means decodes, as encoded data of said first time information, the number of decoded and synthesized signals. changes of bits and a bit string obtained by the repeated bit changes and decodes said first time information by adding a code of a length of a predetermined set value to said bit string by the number of bit changes, and said decoding and synthesizing means synthesize a decoded image signal on the basis of the first and second time information decoded by said time information decoding means.

17. An image decoding device that decodes a bit stream that is formed by coding an image for each object, further characterized by means of decoding time information to decode the first time information defining the time interval between a reference time and a display time and the second time information that defines the display time with higher precision than the time defined by said first time information, as information that defines the display time of an image at each time in a series of images, and an image corresponding to each time; and decoding and synthesizing means for decoding inserted encoded image signals for each object and synthesizing these decoded signal signals, and wherein said time information decoding means retains the first time information of a previously decoded image immediately , then add the first time information of the immediately decoded image of said first time information retention means to a decoded bit stream as the first time information of an image to decode and decode the first time information of the image to be decoded, and said decoding and synthesizing means synthesize the decoded signal signals on the basis of the first and second time information decoded by said time information decoding means.

18. An image coding method, further characterized by: the generation of an encoded image signal, encoding an image for each object based on the predetermined display speed information and the extraction of the encoded image signal and said display speed information after multiplexing them.

19. An image decoding method, further characterized by: decoding a display speed information signal of a coded bit stream that is formed by encoding a picture for each object; and decoding the image by controlling its reconstruction of the encoded image for each object on the basis of the decoded display rate information.