JP2000152234A - Processor and method for data processing, and data processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decode data which are encoded for each of plural image information (object) by a decoder having arbitrary encoding specifications by providing a varying means which varies the number of pieces of image information in a data sequence. SOLUTION: A profile level decision unit of a profile level adjustment part 205 places a header changing unit in operation to change and encode the contents of a profile and level indication(PLI) for a decoder 207 and delete header information regarding an object which is not decoded (discarded) by the decoder 207 according to the comparison result of a variable-code comparator. Namely, the header information of the encoded data is replaced with contents matching the decoding capability of the decoder 207 or an inputted profile and a level. Further, arrangement information regarding the object corresponding to the discarded object is deleted from object arrangement information αto generate new object arrangement information β.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing apparatus and method for processing a data sequence forming one image by using a plurality of encoded image information, and a data processing system.

[0002]

2. Description of the Related Art In recent years, the MPEG4 (Moving Picture Experts Group Phase 4) standard has been standardized as a new coding method for moving images. In the moving picture coding method represented by the conventional MPEG2 standard, coding is performed in units of frames or fields. However, contents (persons, buildings,
MPE to realize reuse and editing of voice, sound, background, etc.)
The G4 standard is characterized in that video data and audio data are treated as objects. Furthermore, the objects included in the video data are also encoded independently, and each can be treated as an object.

FIG. 17 shows a functional block diagram of an encoder based on the MPEG4 standard, and FIG. 18 shows a functional block diagram of a decoder for decoding encoded data by the encoder. In FIG. 17, input image data is an object definition unit 1001.
The objects are divided into respective objects, and the optimal encoding is performed for each of the divided objects, and encoding is performed by respective object encoders 1002 to 1004. Also, information for arranging each object on the decoding side is encoded by an arrangement information encoder 1011. The coded data thus obtained is multiplexed by the multiplexer 1005 and output as one coded data.

When the coded data is input to the decoder shown in FIG. 18, the multiplexed data is first demultiplexed by the demultiplexer 1006 to obtain coded data of each object. The obtained encoded data is decoded by decoders 1007 to 1009 corresponding to each object. At the same time, the arrangement information decoder 1012 decodes the arrangement information of each object. Object decoder 10
The outputs of 07 to 1009 are combined by the combiner 1010 according to the object arrangement information and displayed as an image.

[0005] As described above, according to the MPEG4 standard, various objects can be freely arranged on the decoding side by treating objects in a moving image individually. Also, broadcasting and content creation companies generate encoded data of objects in advance,
It has become possible to generate a great deal of moving image data from limited content.

[0006]

However, as described above, in the encoding system of the MPEG4 standard, an unspecified number of objects are handled, so that the decoding side, in particular, is not sufficient for decoding all the objects. The number of decoding means could not be determined, and thus it was very difficult to construct an apparatus or system.

[0007] Therefore, in the standardized MPEG4 standard, the concept of profile and level is defined, and the profile is composed of profile and level so that specifications can be determined in designing coded data and an encoder / decoder. As encoding specifications, upper limits of the number of objects and the bit rate are provided. FIG. 20 shows an example of a profile table that defines the upper limit of each requirement for each profile level.

[0008] As shown in the profile table of FIG.
In the EG4 standard, the combination of means (tools) used for encoding differs depending on the profile, and the amount of encoded data of the image to be handled is divided stepwise according to the level. Here, both the maximum value of the number of objects that can be handled and the maximum value of the bit rate represent the upper limit in the encoding specification, and if the value is less than that, it is included in the encoding specification. For example, using a tool that can be used with the Core profile, the number of objects is 6, 300 kbps
, The encoded data (encoder)
Is equivalent to level 2.

FIG. 19 shows an example of a bit stream of MPEG4 encoded data. The profile and level described above are the same as profile_and_level_indica in the bitstream.
(PLI in the figure). In MPEG4, the arrangement information of an object is represented and encoded in a system description language, and this information is described at the top for convenience. Actually, they are appropriately multiplexed with other encoding results.

[0010] MPEG4 encoded data is hierarchized from the viewpoint of improving the encoding efficiency and the editing operability. As shown in FIG. 19, a visual_object_sequence_start_code for identification
(VOSSC in the figure), followed by the coded data of each visual object, and finally, visual_object_sequence_end_code (VOSEC in the figure) indicating the end of the coded data
There is. Here, CG data and the like are defined as the visual object in addition to the captured moving image.

For details of the visual object,
Visual_object_start_code for identification (Vi in the figure)
sual Object SC), followed by the aforementioned PLI. Thereafter, is_visual_object_identifier (IVOI in the figure) and visual_
object_verid (VOVID in the figure), visual_object_priority
(VOPRI in the figure), visual_object_type (VOTYPE in the figure), and the like continue to form header information of the visual object. Here, visual_object_type (VOTYPE) is, for example, “0001” when the image is a captured moving image, and is followed by video object (VO) data representing the soul of the encoded data of the moving image. .

The video object data is encoded data representing each object, and is video object layer data for realizing scalability.
(VOL) and video object plane data (VOP) corresponding to one frame of a moving image. Each header part has a code representing the size video_object_layer_width
(VOL_width in the figure), video_object_layer_height (in the figure
VOL_height) and video_object_plane_width (VOP_ in the figure)
width), video_object_plane_height (VOP_heigh in the figure)
t).

A decoder for decoding this bit stream can determine whether decoding is possible or not by referring to the PLI code. That is, decoding cannot be performed in the following cases.

For example, a decoder of Core profile level 1 cannot decode data of Core profile level 2 that exceeds the upper limit of the bit rate or the like.

It is also conceivable to generate encoded data of Simple Profile Level 2 by combining two encoded data of an image which is Simple Profile Level 1 and includes four objects. However, in this case, since the maximum number of objects of level 2 is 4, encoded data that does not belong to any profile or level of MPEG4 will be generated. Therefore, such encoded data cannot be decoded.

Also, for example, a simple profile of 48 kbps
When multiplexing two encoded data of 8 kbps (the number of each object is 2) to generate a new bit stream, the bit rate may not be in 64 kbps. In such a case, you need to set the level to 2,
That is, decoding cannot be performed by the level 1 decoder.

As described above, if the coding specifications (profile and level) of the decoder cannot sufficiently include the coding specifications (profile and level) of the coded data, the coded data is decoded. Could not.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and it is intended that encoded data encoded for each of a plurality of pieces of image information (objects) can be decoded by a decoder having an arbitrary encoding specification. It is an object of the present invention to provide a data processing device and method, and a data processing system.

It is another object of the present invention to provide a data processing apparatus and method capable of adjusting the number of objects included in encoded data, and a data processing system.

[0020]

As one means for achieving the above object, the data processing apparatus of the present invention has the following arrangement.

That is, there is provided a data processing apparatus for processing a data sequence constituting one image by using a plurality of encoded image information, and having a changing means for changing the number of image information in the data sequence. Features.

For example, the changing means includes a detecting means for detecting the number of pieces of image information in the data string, and when the number of pieces of image information detected by the detecting means is larger than a predetermined number, the number of pieces of image information is changed. Is characterized in that

As one means for achieving the above object, the data processing system of the present invention has the following configuration.

That is, encoding means for encoding a plurality of pieces of image information to generate a data string constituting one image, changing means for changing the number of pieces of image information in the data string, Decoding means for decoding the data sequence;
It is characterized by having.

For example, the changing means includes a detecting means for detecting the number of image information in the data string, and the number of image information detected by the detecting means is larger than the number of image information decodable by the decoding means. When the number of image information is large, the number of image information in the data string is reduced.

Further, as one technique for achieving the above object, the data processing method of the present invention includes the following steps.

That is, a data processing method for processing a data sequence constituting one image by a plurality of encoded image information, wherein the number of image information in the data sequence is changed.

For example, a detecting step of detecting the number of image information in the data sequence, a reducing step of reducing the number of image information when the number of detected image information is larger than a predetermined number,
It is characterized by having.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment according to the present invention will be described below in detail with reference to the drawings.

<First Embodiment> FIG. 1 is a block diagram showing a schematic configuration of a moving image processing apparatus according to the present embodiment.
In the present embodiment, MPEG4 is used as the moving picture coding method.
The case where the coding method is used will be described. Note that the encoding method in the present embodiment is not limited to MPEG4, but may be any method as long as a plurality of objects in an image can be respectively encoded.

In FIG. 1, reference numeral 201 denotes an encoder, which captures a moving image and stores a Core profile of an MPEG4 encoding method.
Perform level 2 encoding. 202 is a storage device,
The encoded moving image data is stored. The storage device 202 is constituted by a magnetic disk, a magneto-optical disk, or the like, and is detachable from the present device, so that it can be read by other devices. Reference numeral 203 denotes a transmitter, which performs transmission to a LAN or a communication line, and further performs broadcast and the like. 204 is a receiver,
The encoded data output from the transmitter 203 is received. 205
Is a profile / level adjusting unit to which the present invention is applied. Reference numeral 206 denotes a storage device that stores the output of the profile / level adjustment unit 205. Reference numeral 207 denotes a decoder that can decode encoded data according to the Core profile level 1 of the MPEG4 encoding method. Reference numeral 208 denotes a display for displaying the moving image decoded by the decoder 207. In addition, as described above, the encoder
201 performs encoding according to Core profile level 2, but for ease of explanation, it is assumed that encoding is performed at a bit rate of 384 kbps.

FIG. 15 shows a configuration example of an image to be encoded.
Each symbol in the figure indicates an object.
That is, the object 2000 represents a background, the object 2001 represents a balloon moving in the air, the object 2002 represents a small bird, and the objects 2003 and 2004 represent humans.

FIG. 3A is a diagram showing a bit stream when the image shown in FIG. 15 is encoded.
Object arrangement information α representing position information on the screen of 2000 to 2004 exists. Object placement information α is actually a BIFS (Binary Format for
Scene description) language and multiplexed separately. Then, following the object arrangement information α, the VOSSC code, the visual object data α-
There are 1, α-2, α-3, and VOSEC codes. The encoded data shown in FIG. 3A is stored in the storage device 202 or transmitted via the transmitter 203.

The encoded data is input to a profile / level adjusting unit 205 which is a feature of the present invention via a storage device 202 and a receiver 204. The state of the decoder 207 is also input to the profile / level adjusting unit 205.

FIG. 2 is a block diagram showing a detailed configuration of the profile / level adjusting unit 205. In the figure, reference numeral 1 denotes the encoded data shown in FIG. 2 is a separator,
The encoded data 1 is separated into encoded data representing arrangement information and header information, and encoded data representing each object. Reference numeral 3 denotes a header memory for storing encoded data indicating the separated arrangement information and header information. Code memories 4 to 8 store coded data for each object. Reference numeral 9 denotes a profile / level extractor, which extracts a PLI code from the encoded data 1 and extracts information on a profile and a level. 10 is an object counter,
The number of objects included in the encoded data 1 is counted.

Reference numeral 11 denotes a decoder status receiver.
Get the encoding specification (profile level) and other status of 12 is a profile level input device,
An arbitrary profile or level is set by a terminal (not shown) or the like. 13 is a profile / level determiner,
Profile / level extractor 9 and object counter 1
0 output and the decoder status receiver 11 or profile
It is compared with the profile / level information input from the level input device 12 to determine whether the number of objects needs to be adjusted.

Reference numeral 14 denotes a code length comparator, which encodes data 1
Is input, the code lengths of the objects are counted and compared to determine the code length order of the objects. Reference numeral 15 denotes a header changer which converts the contents of the header information stored in the header memory 3 into a profile / level determiner.
13. Change based on the output of code length comparator 14. Reference numeral 16 denotes a multiplexer, which outputs the output of the header changer 15 and the code length comparator.
The coded data read from the code memories 4 to 8 is multiplexed on the basis of the comparison result of 14. Reference numeral 17 denotes encoded data output as a result of the profile / level adjustment.

Hereinafter, the processing in the profile / level adjusting unit 205 having the above configuration will be described in detail.

The coded data 1 is input to a separator 2, a profile / level extractor 9, an object counter 10, and a code length comparator 14. The separator 2 separates the encoded data 1 into encoded data representing arrangement information and header information, and encoded data representing each object, and each encoded data is divided into a header memory 3 and code memories 4 to 8. Is stored in
For example, the header memory 3 stores the object arrangement information α, VOSSC code, Visual Object SC code, vis
The ual_object_start_code code, each code immediately before the VO data A, the header information of VOL and VOP shown in FIG. 19, and the like are stored. The code memories 4 to 8 store VOL data and VOP data from which header information has been removed for each object. These are stored individually so that the portion from which the header has been removed can be seen. For example, since there are five objects in the image shown in FIG.
Coded data from 0 to 2004 (VO data A to E in Fig. 3 (a))
Are stored in the code memories 4 to 8, respectively.

At the same time, the object counter 10 counts the number of objects included in the encoded data 1.
Then, the code length comparator 14 counts the code length of each object.

The profile / level extractor 9 extracts PLI_α from the encoded data 1 and decodes it to extract information on the profile and level of the encoded data 1. Then, the decoder status receiver 11 is operated at the same time as the extraction, and information such as a profile and a level that can be decoded by the decoder 207 is obtained. These pieces of information can be set by the user via the profile / level input device 12.

The profile / level determiner 13 compares the profile and level information obtained as described above with the extraction result of the profile / level extractor 9 and determines whether the obtained profile and level are encoded data. If it is higher or equal to the profile and level extracted from 1, the header changer 15 does not operate,
The coded data input from the contents of the header memory 3 and the code memories 4 to 8 are sequentially read and multiplexed by the multiplexer 16 to generate coded data 17. That is, the contents of the encoded data 17 in this case are the same as the contents of the encoded data 1.

On the other hand, if the profile and level obtained as a result of the comparison in the profile / level determiner 13 are lower than the profile and level extracted from the encoded data 1, the object counter 10 outputs the encoded data 1 and Input the number of objects included in the
A comparison is made with the number of decodable objects determined from the profile and the level acquired by the level input device 12.

If the number of objects obtained by the object counter 10 is equal to or less than the number of decodable objects, the same as in the case where the obtained profile and level are higher or the same as the coded data 1 described above. Then, the encoded data 17 is generated.

On the other hand, if the number of objects obtained by the object counter 10 is larger than the number of decodable objects, the number of decodable objects is input to the code length comparator 14 to operate the code length comparison function. In the code length comparator 14, a plurality of objects included in the encoded data 1 are set as objects to be decoded in descending order of the code length. That is, decoding is enabled sequentially from the object having the largest code length. For example, in FIG. 3A, the code length of each video object is VO data A, VO data
It is assumed that the order is data D, VO data C, VO data E, and VO data B. Here, in the decoder 207, decoding is performed according to Core profile level 1, so that decoding can be performed up to four objects. Therefore, in the case of FIG. 3A, it can be seen that four objects, that is, four objects except the VO data B can be decoded. Therefore,
The code length comparator 14 disables reading of the code memory 5 storing the VO data B, and the other code memories 4,
6, 7, and 8 can be read.

The profile / level judging unit 13 operates the header changing unit 15 to change and encode the contents of the PLI in accordance with the decoding unit 207, and based on the comparison result by the code length comparing unit 14, decodes the PLI. Deletes the header information relating to the object that is not decrypted (discarded) (VO data B in this case). That is, the header information of the encoded data 1 is replaced with the content suitable for the decoding capability of the decoder 207 or the input profile and level. Further, from the object arrangement information α, the arrangement information relating to the object 2002 corresponding to the discarded object (VO data B) is deleted, and new object arrangement information β is generated.

The header changer 15 and the code memory
The contents of 4, 6, 7, and 8 are read out in the order of input, and multiplexed by a multiplexer 16 to generate encoded data 17. The bit stream of the encoded data 17 at this time is shown in FIG.
According to FIG. 3B, the VOSSC code, the visual object data β-1, β-2, β-3, and the VOSEC code exist after the newly generated object arrangement information β. These visual object data β-1, β-2, β-3 are
The original visual object data α-1, shown in FIG.
The number of objects is adjusted for α-2 and α-3. For example, visual object data β-1
Is composed of a Visual Object SC code, PLI_β representing a profile level suitable for the decoder 207, and a code from which encoded data (VO data B) relating to the object 2002 has been deleted.

The coded data 17 thus obtained is
Is stored in the storage device 206 or decoded by the decoder 207 and displayed on the display 208. FIG. 16 shows encoded data 1
7 shows an image decoded and displayed. According to FIG.
It can be seen that the object 2002 indicating a small bird in the encoding target image shown in FIG.

The code length comparator 14 has been described as counting the code length directly from the coded data 1. However, the code length is calculated based on the coded data stored in the code memories 4 to 8. Is also good.

As described above, according to the present embodiment,
Even when the encoding specifications (profiles and levels) are different between the encoder and the decoder, decoding of the encoded data becomes possible. Further, by discarding the object data having the shortest code length, it becomes easy to select the object to be discarded, and it is possible to minimize the influence on the decoded image.

Further, even when the number of objects that can be decoded by the decoder 207 is smaller than the number specified in the coding specification of the coded data 1, the decoder state receiver 11
The same effect can be obtained by obtaining the number of objects that can be actually decoded.

In addition, even when coded data having a coding rate higher than the coding specification of the decoder 207 is input, the object is discarded so as to reduce the bit rate. Decryption becomes possible.

<Second Embodiment> Hereinafter, a second embodiment according to the present invention will be described.

Note that the schematic configuration of the moving image processing apparatus according to the second embodiment is the same as that of the above-described first embodiment shown in FIG.

FIG. 4 is a block diagram showing a detailed configuration of the profile / level adjusting unit 205 in the second embodiment. 4, the same components as those in FIG. 2 of the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. In the second embodiment, a case will be described in which an MPEG4 encoding method is used as a moving image encoding method, but any encoding method can be applied as long as a plurality of objects in an image can be encoded respectively. It is.

In FIG. 4, reference numeral 18 denotes a size comparator for extracting the size of each object from the header memory 3 and comparing them.

As in the first embodiment, the encoded data 1 is input to a separator 2, a profile / level extractor 9, an object counter 10, and a code length comparator 14, and each encoded data is stored in a header memory 3. , Are stored in the code memories 4 to 8. At the same time, the object counter 10 counts the number of objects included in the encoded data.

The size comparator 18 extracts the size of each object. Here, the size of each object is obtained by extracting and decoding each code of VOL_width and VOL-height shown in the bit stream configuration of the encoded data shown in FIG.

Then, as in the first embodiment, the profile / level extractor 9 extracts information on the profile and the level from the encoded data 1, and at the same time, the decoder status receiver
Information such as the profile and level of the decoder 207 is obtained from 11 or the profile and level are set by the user from the profile / level input unit 12.

The profile / level determiner 13 compares the profile and level information obtained as described above with the extraction result of the profile / level extractor 9 and determines whether the obtained profile and level are encoded data.
If it is higher or equal to the profile and level extracted from 1, the header changer 15 is not operated, and the same encoded data 17 as the encoded data 1 is generated.

On the other hand, if the profile and level obtained as a result of the comparison by the profile / level determiner 13 are lower than the profile and level extracted from the encoded data 1, the object counter 10 outputs the encoded data 1 Input the number of objects included in the
A comparison is made with the number of decodable objects determined from the profile and the level acquired by the level input device 12.

If the number of objects obtained by the object counter 10 is equal to or less than the number of decodable objects, the same as in the case where the obtained profile and level are higher or the same as the coded data 1 described above. Then, the encoded data 17 is generated.

On the other hand, if the number of objects obtained by the object counter 10 is larger than the number of decodable objects, the number of decodable objects is input to the size comparator 18 to operate the size comparison function. The size comparator 18 sets a plurality of objects included in the encoded data 1 as objects to be decoded in descending order of size. That is, decoding can be performed sequentially from the object having the largest size. For example, the size of each object in FIG.
01, 2003, 2002 in order. Here, in the decoder 207, decoding is performed according to Core profile level 1, so that decoding can be performed up to four objects.
Therefore, in the case of the image shown in FIG. 15, it can be seen that the remaining four objects can be decoded except for the smallest object 2002. Therefore, the size comparator 18
Reading of the code memory 5 storing the encoded data of the object 2002 is disabled, and the other code memories are not read.
4, 6, 7, and 8 can be read.

Then, as in the first embodiment, the profile / level judging unit 13 operates the header changing unit 15 to
Is changed according to the content of the decoder 207 and encoded, and based on the comparison result by the size comparator 18, the
Objects not decrypted (discarded) at 207
The header information on (in this case, the object 2002) is deleted. Then, further delete the arrangement information on the discarded object 2002 from the object arrangement information α,
Generate new object arrangement information β.

The header changer 15 and the code memory
The contents of 4, 6, 7, and 8 are read out in the order of input, and multiplexed by a multiplexer 16 to generate encoded data 17. The bit stream of the encoded data 17 at this time is as shown in FIG.

The coded data 17 thus obtained
Is stored in the storage device 206 or decoded by the decoder 207 and displayed on the display 208 as an image as shown in FIG.

Also, the size comparator 18 has been described as extracting the size of the object based on the VOL_width and VOL_height codes of the encoded data 1, but the VOP_widt
The object size may be extracted based on h, VOP_height codes, or based on shape (mask) information obtained by decoding encoded data that actually represents shape (mask) information.

As described above, according to the second embodiment, it is possible to decode encoded data even when encoding specifications differ between an encoder and a decoder. Further, by discarding the object data having the smallest object size, it becomes easy to select the object to be discarded, and it is possible to minimize the influence on the decoded image.

In the first and second embodiments, an example has been described in which one object is discarded. However, it is of course possible to discard two or more objects. It is also possible to configure so that the user can directly specify the object to be discarded.

The order of discarding can be set in advance by the profile / level input unit 12 for each object of the image.

<Third Embodiment> Hereinafter, a third embodiment according to the present invention will be described.

The schematic configuration of the moving picture processing apparatus according to the third embodiment is the same as that of the above-described first embodiment shown in FIG.

FIG. 5 is a block diagram showing a detailed configuration of the profile / level adjusting unit 205 in the third embodiment. 5, the same components as those in FIG. 2 of the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. In the third embodiment, a case will be described in which an MPEG4 encoding method is used as a moving image encoding method, but any encoding method can be applied as long as a plurality of objects in an image can be encoded respectively. It is.

In FIG. 5, reference numeral 20 denotes an object selection indicator, which displays a plurality of objects, and allows the user to select and specify an arbitrary object. twenty one
Is an object selector, and an object selection indicator
An object selector for selecting coded data of an object to be actually processed based on an instruction from 20 and a determination result of the profile / level determiner 13. Reference numerals 22 and 24 denote selectors, which are controlled by the object selector 21 and switch between input and output. An object integrator 23 integrates a plurality of objects. Reference numeral 25 denotes a multiplexer for multiplexing the input encoded data.

As in the first embodiment, the encoded data 1 is input to the separator 2, the profile / level extractor 9, and the object counter 10. The separator 2 separates the encoded data 1 into encoded data representing arrangement information and header information and encoded data representing each object, and the encoded data is divided into a header memory 3 and a code memory 4.
Stored in ~ 8. At the same time, the object counter 10
The number of objects included in the encoded data 1 is counted.

As in the first embodiment, the profile / level extractor 9 extracts information on the profile and the level from the encoded data 1, and at the same time, the decoder status receiver
Information such as the profile and level of the decoder 207 is obtained from 11 or the profile and level are set by the user from the profile / level input unit 12.

The profile / level judging unit 13 compares the profile and level information obtained as described above with the extraction result of the profile / level extractor 9 and determines whether the obtained profile and level are encoded data.
If it is higher or equal to the profile and level extracted from 1, the object selector 21 selects the selector 22
And the path directly connecting 24. That is, the encoded data is prevented from passing through the object integrator 23. Then, without operating the header changer 15, the header memory 3,
The coded data stored in the code memories 4 to 8 are read out in the order of input and multiplexed by the multiplexer 25 to generate coded data 26 similar to the coded data 1.

On the other hand, if the profile and level obtained as a result of the comparison by the profile / level judging unit 13 are lower than the profile and level extracted from the encoded data 1, the object counter 10 sends the encoded data 1 Input the number of objects included in the
A comparison is made with the number of decodable objects determined from the profile and the level acquired by the level input device 12.

If the number of objects obtained by the object counter 10 is equal to or less than the number of decodable objects, the same as in the case where the obtained profile and level are higher or the same as the coded data 1 described above. Then, the encoded data 26 is generated.

On the other hand, if the number of objects obtained by the object counter 10 is larger than the number of decodable objects, the number of decodable objects is
Enter 21. The object selector 21 displays the state of each object (for example, the image shown in FIG. 15), information on each object, and information such as the number of objects to be integrated on the object selection indicator 20. The user selects an object to be integrated according to the information, and gives an instruction to the object selection indicator 20.

Here, the decoder 207 in the third embodiment is
In order to perform decoding by Core profile level 1,
The number of objects that can be decrypted is up to four. Therefore,
For example, since the image shown in FIG. 15 has five objects, by combining two of them into one object, encoded data that can be decoded by the decoder 207 can be obtained. Hereinafter, an example in which the user instructs to integrate the object 2003 and the object 2004 in the image illustrated in FIG. 15 will be described.

The user selects the object selection indicator 20
When the object to be integrated is specified via the, the profile / level determiner 13 operates the header changer 15 to change the contents of the PLI according to the decoder 207, and further selects the result of the selection by the object selector 21. , Header information relating to a new object obtained by integration and deletion of header information relating to an object discarded by integration are performed. Specifically, object 20
Based on the object arrangement information of 03 and 2004, new object arrangement information obtained as a result of integration is generated, and the original object 2003 and 2004 object arrangement information is deleted. Then, based on the header information of the objects 2003 and 2004, the size and other information of the object obtained by integration are generated as header information, and the header information of the original objects 2003 and 2004 is deleted.

The object selector 21 selects the object 20
For the encoded data of 03 and 2004, the object integrator 23 performs an integration process described later, and for the other encoded data, the input / output of the selectors 22 and 24 is controlled so as not to pass through the object integrator 23.

Then, the header changer 15 and the object
Code memories 4, 5, which store coded data of 2000 to 2002
The contents of 6 are read out in the input order and multiplexed by the multiplexer 25 via the selectors 22 and 24. On the other hand, the contents of the code memories 7 and 8 storing the code data of the objects 2003 and 2004 to be integrated are input to the object integrator 23 via the selector 22.

FIG. 6 is a block diagram showing a detailed configuration of the object integrator 23. In the figure, reference numerals 50 and 51 denote code memories which store coded data of objects to be integrated. 52 and 54 are selectors, which switch input and output for each object. An object decoder 53 decodes encoded data and reproduces an object image. Reference numerals 55 and 56 denote frame memories for storing reproduced images for each object. 57 is a synthesizer,
The objects are synthesized according to the arrangement information of the objects to be integrated stored in the header memory 3. An object encoder 58 encodes and outputs the image data obtained by the synthesis.

Hereinafter, the operation of the object integrator 23 will be described in detail. The coded memories 50 and 51 store coded data of the objects 2003 and 2004 to be integrated, respectively. First, the selector 52 selects an input on the code memory 50 side, and the selector 54 selects an output on the frame memory 55 side. Thereafter, the encoded data is read from the code memory 50 and decoded by the object decoder 53, and then the image information of the object 2003 is written to the frame memory 55 via the selector 54. The image data of the object 2003 includes image data representing a color image and mask information representing a shape. Subsequently, the input / output of the selectors 52 and 54 is switched to the other side, and the same processing is performed, so that the image information of the object 2004 is stored in the frame memory 56.

The synthesizer 57 acquires the position information and the size information of the objects 2003 and 2004 from the header memory 3, and
The size of the new object after integration and the relative positions of the original objects 2003 and 2004 in the new object can be obtained. Then, the information in the frame memories 55 and 56 is read, and the color image information and the mask information are combined. FIG. 9 shows the synthesis result of the color image information, and FIG. 10 shows the synthesis result of the mask information.
These color image information and mask information are encoded by the object encoder 58 according to the MPEG4 object encoding scheme, and then output from the object integrator 23.

The encoded data output from the object integrator 23 is multiplexed with other encoded data by the multiplexer 25 via the selector 24 to obtain encoded data 26. Fig. 7
Shows a bit stream of the encoded data 26. FIG. 7 shows the result of performing the integration process of the third embodiment on the encoded data 1 shown in FIG. 3 (a). According to FIG. 7, following the object arrangement information γ including the object arrangement information newly obtained as the integration result, the VOSSC code, the visual object data γ-1, γ-2, γ-3, and VOSE
C code exists. The visual object data γ-1, γ-2, γ-3 are subjected to object integration adjustment with the original visual object data α-1, α-2, α-3 shown in FIG. It was done. For example, the visual object data γ-1 is a PLI representing a profile level suitable for the decoder 207 following the Visual Object SC code.
_γ, V which is each encoded data of objects 2000 to 2002
O data A, VO data B, VO data C, and object 20
Coded data VO data G obtained by integrating 03 and 2004
It consists of.

The coded data 26 thus obtained
Are stored in the storage device 206 or decoded by the decoder 207 to be restored as the image shown in FIG.

In the third embodiment, an example has been described in which the user selects and instructs an object to be integrated in an image using the object selection indicator 20, but the present invention is not limited to this example. . For example, first, the order of integration is set in advance by the object selection indicator 20 for each object of the image. When the number of objects that can be decoded by the decoder 207 is smaller than the number of objects in the image and the need for object integration arises, the object integration can be automatically performed in the set order. It is.

As described above, according to the third embodiment, it is possible to decode encoded data even when the profiles and levels are different between the encoder and the decoder. Further, by integrating and decoding the objects, it is possible to prevent loss of the objects after the decoding.

Further, instead of the object selection indicator 20 and the object selector 21, an object integrator is
By providing the code length comparator 14 and the size comparator 18 and the like shown in the second embodiment, it is possible to perform the integration processing in the order of the object having the shorter code length or the object having the smaller size.

<< Modified Example >> FIG. 8 is a block diagram showing a modified configuration example of the object integrator 23 in the third embodiment. 8, the same components as those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted. FIG. 8 is characterized in that a code length counter 59 is further provided. Code length counter 59
, The code length of the coded data of each object before integration is counted, and the object coder 58 is output so that the code length of the output of the object coder 58 becomes the same as the coefficient result.
(For example, a quantization parameter, etc.) are adjusted. This makes it possible to perform object composition without increasing the overall code length.

<Fourth Embodiment> Hereinafter, a fourth embodiment according to the present invention will be described. The fourth embodiment is characterized in that an object integration process is performed as in the third embodiment. Note that the schematic configuration of the moving image processing apparatus in the fourth embodiment and the detailed configuration of the profile / level adjustment unit 205 are the same as those in FIGS. 1 and 5 in the first and third embodiments described above. Omitted.

FIG. 11 is a block diagram showing a detailed configuration of the object integrator 23 in the fourth embodiment. 11, the same components as those in FIG. 6 of the third embodiment are denoted by the same reference numerals, and description thereof will be omitted.

In FIG. 11, reference numerals 60 and 61 denote separators, which separate input coded data into coded data relating to mask information representing a shape and coded data representing color image information, and output them. . Reference numerals 62, 63, 64, and 65 denote code memories. The code memories 62 and 64 store coded data representing color image information, and the code memories 63 and 65 store coded data related to mask information for each object. . 66
Is a color image information code synthesizer for synthesizing encoded data representing color image information as encoded data. Reference numeral 67 denotes a mask information code synthesizer that synthesizes encoded data representing mask information.

Hereinafter, the operation of the object integrator 23 in the fourth embodiment will be described in detail. As in the third embodiment, first, the objects 2003, 2003 are stored in the code memories 50, 51.
2004 encoded data are stored. Code memory
The coded data of the object 2003 stored in the 50 is read out in frame units (VOP units), separated into color image information coded data and mask information coded data by the separator 60, and Are code memories 62 and 63
Is stored in Similarly, the color image information coded data and the mask information coded data of the object 2004 are stored in the code memories 64 and 65, respectively.

After that, the color image information code synthesizer 66
The color image information encoded data is read from the code memories 62 and 64, respectively. Further, similarly to the third embodiment, the position information and the size information of the objects 2003 and 2004 are acquired from the header memory 3, and the size of the new object after integration and the original objects 2003 and 2004 in the new object are obtained. Find the relative position of each. That is, the color image information code synthesizer 66 performs synthesis on the assumption that when the color image information encoded data is integrated and then decoded, an image as shown in FIG. 9 is obtained as one object.

Here, the MPEG4 encoding system has a data structure called a slice, and a plurality of macroblocks can be defined as one soul that is continuous in the main scanning direction. FIG. 12 shows an example in which the slice structure is applied to the object shown in FIG. In FIG. 12, a region surrounded by a thick frame is defined as one slice, and the leading macroblock is indicated by hatching for each slice.

As shown in FIG. 12, the color image information code synthesizer 66 reads data in the right direction (main scanning direction) in order from the macroblock data corresponding to the upper left of the image obtained as the integration result. That is, first, among the encoded data of the object 2003, the encoded data corresponding to the first macroblock of the first slice is read from the code memory 62. Then, after adding the header information of the slice, the encoded data of the first macroblock is output as it is, and then reading and outputting of the rightward macroblock are sequentially repeated during the slice.

Note that a portion where data newly occurs between the objects 2003 and 2004 is also considered as a new slice. Since this portion is a portion that is not displayed even if decoded by the mask information, an appropriate pixel is supplemented. That is, it is assumed that these portions are composed of only the DC component of the last macroblock including the object. Then, since the DC difference is 0 and the AC coefficients are all 0, no sign is generated.

Then, assuming that a new slice has started at the boundary of the object 2004, the macroblock indicated by hatching in FIG. 12 is set as the head of the new slice, and header information of the slice is added. In this case, since the address of the first macroblock is a relative address, it is converted to a relative address from the macroblock including the previous object. If a macroblock predicts DC or the like with reference to another macroblock,
The part is re-encoded, and thereafter, the code data of the macro block is output as it is sequentially to the right. That is,
The slice header is added at the boundary of the object, and the prediction of the macroblock at the head of the slice is replaced with the code in the initialized state. The code thus obtained is output to the multiplexer 68
Is output to

In parallel with the operation of the color image information code synthesizer 66, the mask information code synthesizer 67
The mask information encoded data is read from 63 and 65, respectively. Then, from the header memory 3, the objects 2003, 200
The position information and the size information of 4 are acquired, and the size of the new object after integration and the relative positions of the original objects 2003 and 2004 in the new object are obtained. Then, the input mask information encoded data is decoded and combined to obtain a mask image shown in FIG. The mask image is encoded by an arithmetic encoding method which is an MPEG4 shape information encoding method. The code thus obtained is output to multiplexer 68.

However, the encoding of the mask information is performed by MPEG.
4 is not limited to the arithmetic coding method. For example, in the synthesis result of the mask information coded data, only the 0 run between the object boundaries becomes longer. Therefore, by applying a method of encoding the 0 run used in the facsimile apparatus, the mask information code Without performing decoding in the synthesizer 67, synthesis can be performed only by replacing the code representing the 0 run length. In general, when mask information is encoded by an arithmetic coding method or another coding method, a change in the code length is small.

In the multiplexer 68, the encoded data relating to the integrated color image information and the mask information encoded data are multiplexed to form encoded data of one object. Subsequent processing is the same as in the third embodiment described above, and is multiplexed with other encoded data by the multiplexer 25 and output.

As described above, according to the fourth embodiment, it is possible to decode encoded data even when the profiles and levels are different between the encoder and the decoder. Also, by integrating objects as encoded data, only a small amount of header information is added,
Loss of the object after decryption can be prevented.

Furthermore, in the object integration processing in the fourth embodiment, a newly added header can be obtained by a small operation, and the code change is limited to the first block of the slice. Speeding up can be achieved as compared to the object integration processing by decoding and re-encoding shown above.

<Fifth Embodiment> Hereinafter, a fifth embodiment according to the present invention will be described. The fifth embodiment is characterized in that an object integration process is performed as in the third embodiment. Note that the schematic configuration of the moving image processing apparatus according to the fifth embodiment is the same as that of FIG. 1 of the above-described first embodiment, and a description thereof will not be repeated.

FIG. 13 is a block diagram showing a detailed configuration of the profile / level adjusting unit 205 in the fifth embodiment. 13, the same components as those in FIG. 5 of the third embodiment are denoted by the same reference numerals, and description thereof will be omitted. In the fifth embodiment, a case will be described in which an MPEG4 encoding method is used as a moving image encoding method, but any encoding method can be applied as long as a plurality of objects in an image can be encoded. It is.

In FIG. 13, reference numeral 70 denotes an object arrangement information judging unit, which determines an object to be integrated.

The profile / level determiner 13 compares the profile and level information of the decoder 207 with the profile / level of the encoded data 1 as in the third embodiment. Even if the profile and level are higher or the same as or lower than the profile and level of the encoded data 1, the number of objects obtained by the object counter 10 is the number of objects that can be decoded by the decoder 207. If it is less than or equal to the number, the coded data 26 is generated in the same procedure as in the third embodiment.

On the other hand, if the number of objects obtained by the object counter 10 is larger than the number of objects that can be decoded by the decoder 207, the number of objects that can be decoded is input to the object arrangement information determiner 70. here,
As in the third embodiment, the number of objects that can be decoded by the decoder 207 is up to four. Therefore, in the image having five objects shown in FIG. 15, by integrating the two objects, it is possible to obtain decoded data that can be decoded.

The object arrangement information judging unit 70 extracts position information and size information of each object from the header memory 3 and determines two objects to be integrated based on the following conditions. The above conditions are (1),
The priority is given in the order of (2).

(1) One object is included in the other object. (2) The distance between the two objects is shortest. In the image shown in FIG.
Is included in the object 2000. Therefore, in the object arrangement information judgment unit 70, the object 2000
And the object 2001 are determined as integration targets.

When the object to be integrated is determined, the
As in the third embodiment, the profile / level judging unit 13 operates the header changing unit 15 to change and encode the contents of the PLI according to the decoder 207,
Based on the determination result by 70, header information relating to a new object obtained by integration and deletion of header information relating to an object discarded by integration are performed. Specifically, based on the object arrangement information of the objects 2000 and 2001, the new object arrangement information obtained as the integration result is generated, and the original object 2000 and 2001 object arrangement information is deleted.
Then, based on the header information of the objects 2000 and 2001, the size and other information of the object obtained by integration are generated as header information, and the original object 20 is generated.
Delete the header information of 00 and 2001.

The object arrangement determiner 70 performs an integration process on the coded data of the objects 2000 and 2001 in the object integrator 23, and does not pass the other coded data through the object integrator 23.
The input / output of the selectors 22 and 24 is controlled.

The header changer 15 and the object
Code memory 6, 7, which stores the encoded data of 2002-2004
8 are read out in the order in which they are input, and input to the multiplexer 25 via the selectors 22 and 24. On the other hand, the contents of the code memories 4 and 5 storing the code data of the objects 2000 and 2001 to be integrated are integrated by the object integrator 23 via the selector 22 and then input to the multiplexer 25. Then, the multiplexed data is multiplexed by the multiplexer 25, so that the coded data 26 is obtained. Note that the integration process in the object integrator 23 is realized in the same manner as in the above-described third or fourth embodiment.

FIG. 14 shows a bit stream of the encoded data 26 in the fifth embodiment. FIG.
6 shows a result of performing the integration processing of the fifth embodiment on the encoded data 1 shown in FIG. According to FIG. 14, following the object arrangement information δ including the object arrangement information newly obtained as the integration result, the VOSSC code, the visual object data δ-1, δ-2, δ-3, and the VOSEC code are used. Exists. This visual object data δ-1,
δ-2, δ-3 are obtained by subjecting the original visual object data α-1, α-2, α-3 shown in FIG. 3 (a) to object integration adjustment. For example, following the Visual Object SC code, PLI_δ representing a profile level suitable for the decoder 207, encoded data VO data H obtained by integrating objects 2000 and 2001, and objects 2002 to It is composed of VO data C, VO data D and VO data E which are the respective encoded data of 2004.

The encoded data 26 thus obtained is
Are stored in the storage device 206 or decoded by the decoder 207 to be restored as the image shown in FIG.

As in the first or second embodiment, the code length and object size of each object may be added to the integrated object determination condition in the fifth embodiment.

As described above, according to the fifth embodiment, it is possible to decode encoded data even when the profiles and levels are different between the encoder and the decoder.

Further, by integrating based on the positional relationship of the objects, it is possible to prevent the loss of the object after decoding while minimizing the amount of code changed by the integration.

In the fifth embodiment, the example in which the object to be integrated is determined based on the positional relationship between the objects has been described. It is also possible to make a selection based on

In the third to fifth embodiments, an example has been described in which two objects are integrated to generate one object. Of course, three or more objects may be integrated or two or more sets may be integrated. It is also possible to perform integration.

The configurations of the code memories 4 to 8 and the header memory 3 in the above-described first to fifth embodiments are not limited to the example shown in FIG. 2, and more code memories may be provided. One memory may be divided into a plurality of areas and used, or a storage medium such as a magnetic disk may be used.

The selection of the object to be discarded or integrated also includes the object size, code length,
It may be determined by combining a plurality of conditions, such as a positional relationship and a user's instruction.

When the present invention is applied to an image editing apparatus, even if the number of objects changes due to the editing process,
The output can be adapted to any profile or level.

<Other Embodiments> Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), a device including one device (for example, For example, the present invention may be applied to a copying machine, a facsimile machine, and the like.

An object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (or CPU) of the system or the apparatus.
And MPU) read and execute the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD
-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) running on the computer based on the instruction of the program code. ) May perform some or all of the actual processing, and the processing may realize the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that the CPU included in the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments. When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

[0134]

As described above, according to the present invention, encoded data encoded for each of a plurality of pieces of image information (objects) can be decoded by a decoder of any specification.

Further, the number of objects included in the encoded data can be adjusted.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a moving image processing device to which the present invention has been applied.

FIG. 2 is a block diagram illustrating a configuration of a profile / level adjustment unit according to the first embodiment.

FIG. 3 is a diagram showing a configuration example of encoded data of a moving image,

FIG. 4 is a block diagram illustrating a configuration of a profile / level adjusting unit according to the second embodiment.

FIG. 5 is a block diagram illustrating a configuration of a profile / level adjusting unit according to a third embodiment;

FIG. 6 is a block diagram illustrating a configuration of an object integrator according to a third embodiment;

FIG. 7 is a diagram illustrating a configuration example of encoded data after integration according to the third embodiment;

FIG. 8 is a block diagram illustrating a configuration of an object integrator according to a modification of the third embodiment;

FIG. 9 is a diagram illustrating a synthesis example of color image information according to the third embodiment.

FIG. 10 is a diagram illustrating a synthesis example of mask information according to the third embodiment;

FIG. 11 is a block diagram illustrating a configuration of an object integrator according to a fourth embodiment.

FIG. 12 is a diagram illustrating a slice structure of color image information according to a fourth embodiment.

FIG. 13 is a block diagram illustrating a configuration of a profile / level adjusting unit according to a fifth embodiment.

FIG. 14 is a diagram showing a configuration example of moving image encoded data after integration in the fifth embodiment;

FIG. 15 is a diagram illustrating a configuration example of an image represented by encoded data;

FIG. 16 is a diagram illustrating a configuration example of an image represented by encoded data;

FIG. 17 is a diagram illustrating a configuration example of an encoder according to the MPEG4 standard.

FIG. 18 is a diagram illustrating a configuration example of a decoder according to the MPEG4 standard.

FIG. 19 is a diagram illustrating a configuration example of moving image encoded data according to the MPEG4 standard.

FIG. 20 is a profile table according to the MPEG4 standard.

[Explanation of symbols]

 1, 17, 26 Coded data 2, 60, 61, 1006 Separator 3 Header memory 4, 5, 6, 7, 8, 50, 51, 62, 63, 64, 65 Code memory 9 Profile level extractor 10 Object counter 11 Decoder status receiver 12 Profile level input unit 13 Profile level judgment unit 14 Code length comparator 15 Header changer 16, 25, 68, 1005 Multiplexer 18 Size comparator 20 Object selection indicator 21 Object selector 22, 24, 52, 54 Selector 23 Object integrator 53, 1007, 1008, 1009 Object decoder 55, 56 Frame memory 57, 1010 Compositor 58, 1002, 1003, 1004 Object encoder 59 Code length counter 66 Color image information code synthesizer 67 Mask information code synthesizer 70 Object placement information determiner 201 Encoder 202, 206 Storage device 203 Transmitter 204 Receiver 205 Profile / level adjuster 207 Decoder 208 Display 1001 Object definer 1011 Allocation information encoder 1012 Allocation information decoder 2000, 2001, 2002, 2003, 2004 Objects

Continued on the front page F-term (reference) 5B057 BA23 CA01 CA16 CA18 CB01 CB16 CB18 CE08 CG07 5C057 AA06 AA07 CA01 CE10 EL01 EM00 GF07 GG01 GH00 5C059 KK22 KK36 MA00 MB01 MB11 MB12 MB21 ME11 PP04 PP28 PP29 SS26 TC18 SS20 SS18 SS18 UA02 UA05 UA35 5C078 AA09 BA44 BA57 CA14 DA06 DA07 9A001 EE04 HH27 JZ19

Claims (27)

[Claims] 1. A data processing apparatus for processing a data sequence forming one image by using a plurality of encoded image information, comprising a changing unit for changing the number of image information in the data sequence. Characteristic data processing device. 2. The image processing apparatus according to claim 1, wherein the changing unit includes a detecting unit configured to detect a number of image information in the data string, and when the number of the image information detected by the detecting unit is larger than a predetermined number, the number of the image information is changed. 2. The data processing apparatus according to claim 1, wherein the data processing number is reduced. 3. The image processing apparatus according to claim 2, wherein the changing unit reduces the number of image information by discarding the image information.
The data processing device according to claim 1. 4. The image processing apparatus according to claim 3, wherein said changing means includes code length monitoring means for monitoring a code length of each image information in said data sequence, and sequentially discards image information having a shorter code length. The data processing device according to claim 1. 5. The data according to claim 3, wherein said changing means includes a size monitoring means for monitoring a size of each image information in said data string, and sequentially discards image information having a smaller size. Processing equipment. 6. The data processing apparatus according to claim 2, wherein the change unit reduces the number of pieces of image information by integrating a plurality of pieces of image information. 7. The image processing apparatus according to claim 1, wherein the changing unit includes: a selecting unit that selects a plurality of pieces of image information in the data string; and an integrating unit that integrates the plurality of pieces of selected image information. 7. The data processing device according to 6. 8. The data processing apparatus according to claim 7, wherein said selection means performs a manual selection by a user. 9. The integrating means, decoding means for decoding a plurality of pieces of image information selected by the selecting means, synthesizing means for synthesizing the decoded plural pieces of image information, 8. The data processing apparatus according to claim 7, further comprising: an encoding unit that performs encoding. 10. A counting means for counting a code length of image information selected by said selection means, wherein said coding means controls a coding parameter based on a counting result by said counting means. The data processing apparatus according to claim 9, wherein: 11. The separating means for separating the plurality of pieces of image information selected by the selecting means into color information and mask information, respectively, and a color information synthesizing means for synthesizing the separated color information. 8. The data processing apparatus according to claim 7, comprising: mask information combining means for combining the separated mask information; and multiplexing means for multiplexing the combined color information and mask information. 12. The data processing apparatus according to claim 7, wherein said selection means selects a plurality of pieces of image information based on a position of said image information. 13. The apparatus according to claim 1, wherein said selecting means selects a plurality of pieces of image information that are inclusive of each other.
2. The data processing device according to 2. 14. The data processing apparatus according to claim 12, wherein said selecting means selects a plurality of pieces of image information whose distance from each other is equal to or less than a predetermined value. 15. The image processing apparatus according to claim 6, wherein the changing unit includes a code length monitoring unit that monitors a code length of each image information in the data sequence, and sequentially integrates the image information having the short code length. The data processing device according to claim 1. 16. The data according to claim 6, wherein said changing means includes a size monitoring means for monitoring a size of each image information in said data sequence, and sequentially integrates the image information having the smaller size. Processing equipment. 17. The image processing apparatus according to claim 3, wherein the changing unit includes a priority adding unit that adds a priority to each image information in the data sequence, and discards the image information based on the priority. Data processing equipment. 18. The image processing apparatus according to claim 6, wherein the change unit includes a priority adding unit that adds a priority to each image information in the data sequence, and integrates the image information based on the priority. Data processing equipment. 19. The data processing apparatus according to claim 2, wherein the predetermined number is the number of image information that can be decoded based on the output specification of the data string. 20. The data processing apparatus according to claim 19, wherein the data sequence is encoded based on the MPEG4 standard, and the output specification is an encoding specification conforming to the MPEG4 standard. 21. Encoding means for encoding each of a plurality of pieces of image information to generate a data sequence constituting one image; changing means for changing the number of image information in the data sequence; A data processing system, comprising: decoding means for decoding the data sequence. 22. The image processing apparatus according to claim 19, wherein the changing unit includes a detecting unit configured to detect a number of pieces of image information in the data string, wherein a number of the image information detected by the detecting unit is larger than a number of image information that can be decoded by the decoding unit. 22. The data processing system according to claim 21, wherein the number of image information in the data sequence is reduced when the number of image information is large. 23. One of a plurality of encoded image information
A data processing method for processing a data sequence forming one image, wherein the number of image information in the data sequence is changed. 24. A method comprising: detecting a number of pieces of image information in the data sequence; and reducing the number of pieces of image information when the number of pieces of detected image information is larger than a predetermined number. 24. The data processing method according to claim 23, wherein: 25. The data processing method according to claim 24, wherein in the subtraction step, the number of image information is reduced by discarding the image information. 26. The data processing method according to claim 24, wherein in the subtraction step, the number of image information is reduced by integrating the image information. 27. One of a plurality of encoded image information
A recording medium on which a program code of data processing for processing a data sequence constituting one image is recorded, wherein the program comprises: a code of a detection step for detecting the number of image information in the data sequence; If the number of image information is larger than the predetermined number,
A code for a subtraction step for reducing the number of image information.