JP2001094980A - Data processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem where processing for re-coding takes a very long time, and excessive software and hardware units are required because, in order to suppress a bit rate below a target bit rate, an entire bit rate is obtained first after deciding objects to be combined and when the obtained bit rate exceeds the target rate, code data are required to be re-coded by revising a frame rate and a quatization width so that each of decoded objects has a shared bit rate after decoding of the coded data of each object once more after once decoding each object, when the bit rate exceeds the object rate. SOLUTION: When generating new image data through the combinations of a plurality of coded data of dynamic images coded for each object, the code quantity is decreased by sequentially deleting data not referred to from others, in the case of decoding without applying re-coding to the coded data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing for inputting coded data and reducing the coded data to a predetermined code amount or less.

[0002]

2. Description of the Related Art As a conventional moving picture coding technique, MPE
There are international standard systems such as G2, H263, and MPEG4. These methods divide an input image into multiple blocks, perform motion compensation / prediction, orthogonal transform, etc., quantize transform coefficients, and perform variable-length coding using entropy coding. It realizes efficient coding.

[0003] When such moving picture coding is performed, the amount of code generated greatly varies depending on the picture of the input image, the amount of movement of the subject, and the like. In order to keep the generated code amount constant, a method such as controlling the quantization width according to the generated code amount or skipping an input frame (not performing coding) during encoding is used. . Such code amount control is basically performed at the time of encoding.

As a method of controlling the code amount, a method of changing a quantization width when performing quantization, a method of changing a frame rate, and the like are used.

In MPEG-4, when encoding, an input image is divided into a moving part such as a person and a non-moving part such as a background, and image data is encoded for each divided partial image. Has also been done. Each of the divided partial images is called an object.

FIGS. 7 to 9 show examples of image coding using objects. In the examples of FIGS. 7 and 8, three objects 7 are input from the input images 701 and 801 on the encoding side.
02 to 704 and 802 to 804 are extracted, each is encoded and stored as data. The background images 700 and 800 are also encoded as one of the objects and stored as data.

The decoding side decodes and combines all the encoded data 705 to 708 stored as shown in FIG. 7 as well as some object data 805 and 806, as shown in FIG.
An image can also be synthesized by decoding only 808. Alternatively, as shown in FIG. 9, encoded data 90 created on the encoding side is used.
In addition to 5 to 908, by obtaining arbitrary encoded data from another database 909 and decoding and combining on the decoding side,
New image data 910 can also be used.

[0008] By the way, encoded data is transmitted to the decoding side by some means, but the bandwidth of the transmission path varies depending on the apparatus, the number of lines, or the use state of the lines.

In order to transmit coded data with no delay at any bandwidth, the bit rate of the coded data may be set low. However, if the bit rate is lowered, the quality of a moving image is generally lowered. . In order to improve the quality of a moving image to be decoded, it is necessary to make the bit rate of the encoded data large enough so as not to exceed the bandwidth of the line.

In the case of image coding using an object,
The bit rate of the encoded data to be transmitted to the decoding side is the sum of the bit rates of the encoded data of the selected individual objects.

[0011]

As described above, when new image data is created by combining encoded data of a plurality of objects, the overall bit rate is the sum of the bit rates of the individual encoded data. However, there is no guarantee that the overall bit rate will always be controlled below the target rate.

In order to keep the bit rate below the target rate, after determining the objects to be combined, the overall bit rate is determined, and if the target rate is exceeded, an appropriate bit rate for each object is allocated again. There is a need to.

That is, for each object,
After decoding once, decode the encoded data of each object once, change the frame rate and quantization width so that each decoded object has the allocated bit rate, and encode again Will do.

In this case, the process for re-encoding takes a very long time, and furthermore, extra hardware and software are required, which is a problem.

Accordingly, an object of the present invention is to provide an optimal image data without re-encoding when generating new image data by combining a plurality of encoded data of moving images encoded for each object. To convert to bit rate.

[0016]

According to a first aspect of the present invention, there is provided an apparatus for forming a desired image by multiplexing one or more partial images having an arbitrary shape. An encoded data input unit for obtaining partial image encoded data obtained by encoding a partial image, and an encoded data processing unit for reducing a code amount by deleting predetermined encoded data in the inputted partial image encoded data And a code amount for detecting the code amount of each partial image coded data, controlling the coded data processing unit, and managing a code allocation amount and a total code amount for each partial image coded data forming an image. The above problem is solved by providing a controller and deleting a part of the input partial image encoded data to reduce the total code amount after multiplexing to a predetermined code amount or less.

According to the second aspect of the present invention, in the code amount controller, visual features such as motion, color, and size of each partial image are extracted, and each visual feature is extracted based on the visual features. The above problem is solved by determining the code amount to be allocated to the partial image.

According to a third aspect of the present invention, the coded data is subjected to inter-frame predictive coding, and the coded data processing unit deletes frames in order from a frame which is not referred to by others. The above problem is solved by reducing the amount of code by performing.

According to a fourth aspect of the present invention, the encoded data has a hierarchical structure in which decoding is performed by using decoding information of a lower layer when decoding an upper layer. In the coded data processing unit, the above problem is solved by reducing the amount of codes by deleting coded data in order from a higher layer.

[0020]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows a schematic diagram of the present invention. 1 and 2, the number of objects is described for explanation.
Although the number is 3, the present invention is applicable to other numbers of objects. The input image is first divided into objects by the object extracting unit 100. Each object is passed to the object encoding units 101 to 103 and is subjected to predetermined encoding. Also for the coding here, a predetermined coding amount control is performed, but the coding amount control here is not related to the present application and will not be described.

The coded data output from the object coding units 101 to 103 are passed to coded data processing units 104 to 106, where the code amount is converted. Encoded data processing units 104 to 106
Are controlled by the controller 107. The controller 107 controls the coded data processing units 104 to 106 based on the set target bit rate and the generated code amount of each object sent from the coded data processing units 104 to 106.

The coded data of each object whose code amount has been converted by the coded data processing units 104 to 106 is supplied to the multiplexing unit 10.
It is multiplexed by 8 and sent to the decoding unit.

FIG. 2 shows an example in which the present invention is applied to a case where encoded data which has already been encoded in object units is stored in a database 200. This is because the parts from the input image shown in FIG. 1 to the object encoding units 101 to 103 are stored in a database of the object encoded data.
Replaced with 200. The other parts are the same as those in FIG. 1 described above, and the description will not be repeated.

The present invention is also applicable to a case where FIGS. 1 and 2 are combined. That is, the input image is divided into object units by the object extraction unit, and the input image is arbitrarily selected from encoded data encoded by the object encoding unit and encoded data already encoded and stored in the database of the object encoded data. And the coded data is converted by the coded data processing unit and multiplexed by the multiplexing unit.

Next, a first embodiment of the present invention will be described. Normally, inter-frame prediction is performed for encoding a moving image, but depending on the difference in the prediction direction or the presence or absence of prediction.
There are three picture types. FIG. 10 shows this state. The solid-line and broken-line arc arrows in the figure indicate the prediction direction of each frame. FIG. 10 shows the frames arranged in the order of reproduction, and the encircled numerals in the frames are the order of encoding (the order of decoding).

The picture type includes an I picture (frames 1 and 8 in FIG. 10) which is independently encoded without depending on other frames, and a P which performs prediction from the immediately preceding I picture or P picture. There are pictures (frames 2, 4, and 6 in FIG. 10) and B pictures (frames 3, 5, 7, and 9 in FIG. 10) for which prediction is performed based on temporally preceding and succeeding I or P pictures.

As described above, since the B picture is predicted in both directions, the encoding order is different from the reproduction order, and the frame to be referred to is encoded first.

In the first embodiment, the amount of code is reduced by deleting a frame. In this case, it is necessary to delete coded data of a frame which, even if deleted, does not affect decoding of other frames. There is.

For example, in the case of FIG. 10, the B picture (the third, fifth, seventh and ninth frames in FIG. 10) can be deleted because it does not affect the decoding of other frames even if the B picture is deleted. After that, the P picture (the sixth frame in FIG. 10) immediately before the I picture can also be deleted.

In the present embodiment, in the image encoding device or the encoded data processing device composed of FIG. 1, FIG. 2 or a combination thereof, the encoded data processing units 104 to 106 and 201 to 203, which perform code amount conversion, Apply coded data processing using frame thinning. Hereinafter, details of the coded data processing unit by frame thinning shown in FIG.
Controller for controlling the encoded data processing unit shown in FIG.
Details of 107 and 204 will be described.

FIG. 3 is a block diagram of a coded data processing unit for reducing the input object coded data by frame thinning. Switch 1 (300), 2
(303) changes the flow of encoded data according to an instruction from the controller, and the generated code amount monitoring unit 301 measures the code amount of the transmitted encoded data and notifies the controller of the code amount. It is. The buffer 302 is a place for temporarily storing encoded data, and the frame thinning unit 304 is a part for deleting a frame that can be deleted from the encoded data.

The object encoded data is first taken into the generated code amount monitoring unit 301 via the switch 1 (300) according to an instruction from the controller. The generated code amount monitoring unit 301 measures the code amount of the encoded data, transmits the information to the controller, and sends the encoded data to the buffer 302. The encoded data sent to the buffer 302 is sent to the frame thinning unit 304 via the switch 2 (303) when the controller determines that the code amount needs to be reduced, and the code amount needs to be reduced. If it is determined that there is no data, it is sent to the multiplexing unit via the switch 2 (303). The encoded data sent to the frame thinning unit 304 is reduced in codes by frame thinning and sent to the generated code amount monitoring unit 301 via the switch 1 (300). Generated code amount monitoring unit 301
The encoded data sent to is checked again for the code amount,
If necessary, data reduction is performed repeatedly.

The frame thinning unit 304 receives an instruction from the controller and executes frame deletion. The procedure will be described with reference to FIG. First, the amount of code to be allocated to each object coded data is determined based on an input from the controller (1100). Next, it is checked whether or not each object encoded data amount is equal to or less than the assigned code amount (1101). If the coded data amount is larger than the allocated code amount, it is first checked whether or not a B picture exists in the coded data (1102).
The picture is deleted (1105). Taking FIG. 10 as an example, 3,
Delete any frame from frames 5, 7, and 9. Here, only one frame is deleted. Each time the B picture is reduced by one frame, the generated code amount monitoring unit 301 checks the generated code amount again to confirm whether the code amount of the object coded data is equal to or less than the allocated code amount (1101). If the code amount is not less than the allocated code amount, frame thinning is performed again. When performing frame thinning, first, it is checked whether or not a B picture exists in encoded data (1102). If there is no B picture to be deleted, deletion of a P picture is attempted. To delete a P picture, first check whether there is a P picture in the encoded data (1103), and if so, delete it sequentially from the P picture immediately before the I picture so as not to affect decoding of other frames (1106). In the example of FIG. 10, 6
No. 4, No. 4, and No. 2 frames. Each time the P picture is reduced, the generated code amount monitoring unit 301 checks the generated code amount again as in the case of the B picture, and confirms whether the code amount of the object coded data is equal to or less than the allocated code amount. (1101). If there are no more B pictures or P pictures that can be deleted, any I picture, 8, 1 in FIG.
The number frame is deleted (1104).

In the above example, each time one frame is deleted, it is determined whether or not the number of allocated codes is equal to or less than the assigned code amount.
It is also conceivable to configure so that a plurality or all of the B pictures that can be deleted are deleted at once. The same applies to P pictures and I pictures.

FIG. 4 illustrates a controller for controlling the encoded data processing section. The controller unit allocates a code amount to each coded data according to the set target bit rate, a code amount allocating unit 400, and codes to be reduced from the allocated code amount and the generated code amount input from the coded data processing unit. Reduction code amount determination unit 40 for calculating the amount
1. It is composed of a data reduction instructing unit 402 for issuing an instruction for data reduction to the encoded data processing unit according to the determined data reduction amount.

The code amount allocating unit 400 determines the code amount to be allocated to each object based on the set target bit rate. Allocation of the code amount to each object is performed by a method such as dividing by the number of objects to be combined with the target bit rate, or allocating in proportion to each code amount of the object to be combined with the target bit rate.

Distributing the allocated code amount in proportion to each code amount means, for example, that when a code amount before conversion of object data is 10 kbps, 15 kbps, 15 kbps, and 20 kbps, a given target bit rate is set for each object. 2 to 3 to 3
It means to distribute at a ratio of 4 to 4. When code amount allocation is performed by this method, the code amount input from the encoded data processing unit to the reduced code amount determination unit 401 in FIG. 4 is also input to the code amount allocation unit 400.

When the code amount to be allocated to each object is determined, the code amount to be reduced is calculated from the coded data by the reduced code amount determining unit 401. The data reduction amount is determined by the difference between the code amount allocated by the code amount allocating unit 400 and the code amount input from the coded data processing unit.

The data reduction instructing unit 402 issues a data reduction instruction to the encoded data processing unit when it is necessary to reduce the data, and sends the encoded data to the encoded data processing unit when there is no need for data reduction. Send an instruction to send to the multiplexing unit. These instructions are individually given to the processing units of each encoded data.

As described above, according to the first embodiment,
When one piece of coded data is configured by combining data coded in object units, the data amount of the coded data can be reduced without re-encoding by thinning out the data in frame units from the coded data of each object. It becomes possible to control.

Next, a second embodiment of the present invention will be described.
FIG. 12 and FIG. 13 explain the hierarchy of encoded data. Encoded data having a hierarchical structure is divided into a portion having basic layer information and a portion having enhancement layer information as shown in FIG.

FIG. 13 shows how the encoded data 1300 having a hierarchical structure is decoded. The encoded data is first input to the layer separating unit 1300, and is divided into encoded data for each layer. The encoded data of each separated layer is
Decoding is performed from the lower layer. First, the base layer is decoded (1303). Next, the enhancement layer A (1302) is decoded when a higher-quality decoded image is required, and the enhancement layer B (1301) is decoded when a higher-quality decoded image is required. When decoding the encoded data of the upper layer, the upper layer is decoded using the decoded image of the lower layer. If the upper layer is not decoded, the upper layer information becomes unnecessary, and the information of the layer may be deleted from the encoded data shown in FIG.

In this embodiment, in the image encoding apparatus or the encoded data processing apparatus composed of FIG. 1 or FIG. 2 or a combination thereof, the encoded data processing units 104 to 106 and 201 to 203 perform code amount conversion. , Coded data processing using the hierarchical structure of data is applied. Hereinafter, the encoded data processing unit shown in FIG. 5 will be described. The controller that controls the coded data processing unit is the same as in the first embodiment, and a description thereof will not be repeated. Note that the hierarchical structure has a spatial hierarchy, a temporal hierarchy, and an SNR hierarchy due to differences in properties, and the present embodiment can be applied to all of them.

FIG. 5 illustrates a method for reducing the input object coded data by using the hierarchical structure of the data. The switches 1 (500) and 2 (503) change the flow of encoded data according to an instruction from the controller, and the generated code amount monitoring unit 501 measures the code amount of the transmitted encoded data, and This is the part that notifies the controller of the amount. The buffer 502 is a place where the encoded data is temporarily stored, and the upper layer deletion unit 504 is a unit that deletes a removable upper layer from the encoded data.

The object coded data is first taken into the generated code amount monitoring unit 501 via the switch 1 (500) according to an instruction from the controller. The generated code amount monitoring unit 501 measures the transmitted code amount, transmits the information to the controller, and transmits the encoded data to the buffer 502. The encoded data sent to the buffer 502 is sent to the upper layer removing unit via the switch 2 (503) when the controller determines that the code amount needs to be reduced, and the code amount needs to be reduced. If it is determined that there is no data, it is sent to the multiplexing unit via the switch 2 (503). The encoded data sent to the upper layer deletion unit 504 is reduced in code by deleting the upper layer, and the generated code amount monitoring unit
Sent to 501. The coded data sent to the generated code amount monitoring unit 501 is checked again for the code amount, and if necessary, the data is repeatedly reduced.

The upper layer deletion section 504 receives an instruction from the controller and executes the deletion of the upper layer. The procedure will be described with reference to FIG. First, an input code amount for each object encoded data is determined based on an input from the controller (1400). Next, it is checked whether or not each object encoded data amount is equal to or less than the assigned code amount (1401). If the encoded data amount is larger than the allocated code amount, it is checked whether or not layer information other than the base layer is included in the encoded data (1402). If information of an upper layer exists in the encoded data, the uppermost layer information is sequentially deleted from the encoded data (140).
3). After deleting the upper layer, the code amount is checked again to confirm whether the code amount is equal to or less than the allocated code amount (1401). If the code amount is not less than the allocation amount, the deletion of the upper layer is repeated. If there is no upper layer to be deleted (1404), the code amount may be reduced by frame deletion described in the first embodiment.

As described above, according to the second embodiment,
When one piece of encoded data is formed by combining data encoded in units of objects, the encoded data of each object having a hierarchy is deleted sequentially by removing the upper layer from the encoded data without re-encoding. Can be controlled.

Next, a third embodiment of the present invention will be described. FIG. 6 is a block diagram illustrating a controller according to the present embodiment. The controller of FIG.
4, a reduced code amount determining unit 601, a data reduction instructing unit 602, a code amount allocating unit 600, a feature amount extracting unit 604 for extracting a feature amount of an object from encoded data, A priority determining unit 603 for increasing or decreasing the amount of code to be allocated to each object based on the information is provided.

The feature extracting unit 604 extracts information such as the amount of motion, color, and area of the object from the encoded data, and sends the information to the priority determining unit 603. The priority determination unit 603 determines which object is preferentially assigned a code amount based on the received feature amount of each object.

For example, a small amount of code is allocated to an object that does not move much, such as a background, and is applied to a visually important object such as a moving object such as a person or a skin-colored object such as a human face. Allocate a large amount of code. The result is sent to code amount allocating section 600 in the form of a weighting coefficient at the time of code amount allocation.

The code amount allocating section 600 determines the code amount to be allocated to each object based on the set target bit rate and the weighting coefficient received from the priority determining section 603. For example, for three objects A, B, and C including an object B that is visually important such as a person,
In the controller shown in FIG. 4, even if the code amount is allocated at a ratio of 4: 6: 10, the controller shown in FIG. 6 suppresses the code amount allocated to the objects A and C and allocates a larger code amount to the object B. Such weighting factors work. Therefore, for example, the weights -1, 3, and -2 act on the code amount allocation ratio 4: 6: 10, and the code amounts are allocated to the objects A, B, and C at a ratio of 3: 9: 8. Become like That is, if the target bit rate of the entire object is 100 kbps, the controller of FIG.
Although a code amount of 0 kbps is assigned, the controller shown in FIG.
Code amounts of 15 kbps, 45 kbps, and 40 kbps are assigned to A, B, and C.

When the code amount to be allocated to each object is determined, the code amount to be reduced is calculated from the encoded data by the reduced code amount determination unit 601. The data reduction amount is determined by the difference between the code amount allocated by the code amount allocating unit 600 and the code amount input from the coded data processing unit. The data reduction instructing unit 602 issues a data reduction instruction to the encoded data processing unit when it is necessary to reduce the data, and multiplexes the encoded data to the encoded data processing unit when there is no need for data reduction. To send to

Note that, in addition to the priority information from the priority determining unit 603, the code amount allocating unit 600 allows a user to directly input an object to the object from the outside irrespective of the feature amount of the object, such as the amount of motion, color, and area. A weighting coefficient may be set.

As described above, according to the third embodiment,
When reducing data from object encoded data in the first and second embodiments, while reducing data of other objects while keeping data of visually important objects as much as possible, the overall code amount can be adjusted. It is possible to do.

Next, a fourth embodiment of the present invention will be described. This is obtained by replacing the encoded data processing unit in FIG. 3 in the first embodiment with another configuration.

In this embodiment, a coded data processing unit for performing code amount conversion in an image coding device or a coded data processing device composed of FIG. 1, FIG. 2 or a combination thereof.
Encoded data processing using frame thinning is applied to 104 to 106 and 201 to 203. Hereinafter, the encoded data processing unit shown in FIG. 15 will be described. The controller that controls the coded data processing unit is different from that of the first embodiment only in that the output destination of the control signal from the data reduction instructing unit is changed, and a description thereof will be omitted.

FIG. 15 is a block diagram of a coded data processing unit for reducing the input coded data of an object by frame thinning. The generated code amount monitoring unit 1500 measures the code amount of the transmitted encoded data,
This is a part for notifying the code amount to the controller. buffer
1501 is a place to temporarily store the encoded data,
The frame thinning unit 1502 is a unit that deletes a frame that can be deleted from the encoded data. The thinning frame determining unit is a unit that determines which frame data is to be deleted from the encoded data based on the allocated code amount set by the controller.

The object coded data is first taken into the generated code amount monitoring section 1500. Generated code amount monitoring unit 1500
Measures the amount of code of the coded data, transmits the information to the controller, and stores the coded data in the buffer 1501,
The data is sent to the thinned frame determining unit 1503. The encoded data stored in the buffer 1501 is sent to the frame thinning unit 1502. The thinned frame determining unit 1503 determines which frame should be deleted based on the allocated code amount set by the controller. More specifically, the thinned-out frame determination unit 1503 checks the frame type of each frame in the encoded data transmitted from the generated code amount monitoring unit 1500 and all the code amounts included in the frames. Then, at the time when the assigned code amount is set by the controller, it is determined which frame in the encoded data is combined with the set code amount or less, and a deleted frame is determined. At this time, the method of selecting a frame that can be deleted is the same as in the first embodiment.

The frame thinning section 1502 performs a thinning frame determining section based on the encoded data sent from the buffer 1501.
The frame specified by 1503 is deleted at a time, and the coded data after frame thinning is sent to the multiplexing unit.

In the first embodiment, each time one or one frame is deleted, it is determined whether or not the allocated code amount is equal to or less than the allocated code amount. After all frames to be deleted are determined from the code amount, deletion is performed.

In FIG. 10, when the B picture is deleted, the third, fifth, seventh, and ninth frames are deleted in an arbitrary order in the first embodiment. However, in the fourth embodiment, the allocated code length is deleted. After determining the difference between the number of frames and the generated code amount and determining which frame to delete, the frames are deleted collectively.

As described above, according to the fourth embodiment,
When one piece of coded data is configured by combining data coded in object units, the data amount of the coded data can be reduced without re-encoding by thinning out the data in frame units from the coded data of each object. It becomes possible to control. In the present embodiment, unlike the first embodiment, a necessary amount of encoded data can be deleted at a time without repeating deletion of encoded data and checking of the generated code amount.

In the fourth embodiment, an example of frame thinning has been described. However, even in the case of a method for reducing encoded data using a data hierarchical structure as shown in FIG.
This is possible by replacing the encoded data processing unit with a block diagram as shown in FIG. That is, the method described in the fourth embodiment is effective also in the second embodiment.

That is, as shown in FIG. 12, the difference between the allocated code amount and the generated code amount is calculated for the coded data including the base layer, the enhancement layer A, and the enhancement layer B. If it is deleted, it is determined whether or not the code amount becomes equal to or less than the allocated code amount. For example, by deleting the enhancement layer A and the enhancement layer B collectively, it is not necessary to perform the loop processing as shown in FIG.

[0065]

As described above, according to the present invention, when one bit stream is formed by combining data coded on an object basis, the code amount is larger than the set target bit rate. Even in this case, it is possible to reduce the code amount without re-encoding the encoded data. Therefore, no extra hardware for encoding is required to adjust the code amount, and the processing time is shortened.

According to the first embodiment, when one piece of encoded data is formed by combining data encoded in units of objects, the data is thinned out in units of frames from the encoded data of each object, so that It is possible to control the code amount of the encoded data without encoding.

According to the second embodiment, when one piece of encoded data is formed by combining data encoded in units of objects, the upper layer is sequentially deleted from the encoded data of each object having a hierarchy. By doing
It is possible to control the code amount of the encoded data without re-encoding.

According to the third embodiment, when data is reduced from the object coded data in the first and second embodiments, data of other objects is kept as much as possible while leaving data of visually important objects as much as possible. By reducing the amount, it is possible to adjust the entire code amount, and it is possible to obtain a visually excellent decoded image.

According to the fourth embodiment, when one piece of encoded data is formed by combining data encoded in units of objects, the data is thinned out in units of frames from the encoded data of each object, so that It is possible to control the code amount of the encoded data without encoding. Further, a necessary amount of encoded data can be deleted at a time without repeating the deletion of the frame and the check of the generated code amount.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an embodiment according to the present invention.

FIG. 3 is a block diagram illustrating a configuration of an encoded data processing unit.

FIG. 4 is a block diagram illustrating a configuration of a controller unit.

FIG. 5 is a block diagram illustrating a configuration of an encoded data processing unit.

FIG. 6 is a block diagram illustrating a configuration of a controller unit.

FIG. 7 is a diagram illustrating encoding and decoding processing in a general object unit.

FIG. 8 is a diagram illustrating encoding and decoding processing in a general object unit.

FIG. 9 is a diagram illustrating a general object-based encoding and decoding process.

FIG. 10 is a diagram illustrating a prediction relationship of frames in inter-frame prediction coding.

FIG. 11 is a diagram illustrating encoded data reduction in inter-frame predictive encoding.

FIG. 12 is a diagram illustrating a data example of encoded data having a hierarchical structure.

FIG. 13 is a diagram showing processing when decoding encoded data having a hierarchical structure.

FIG. 14 is a diagram illustrating encoded data reduction of encoded data having a hierarchical structure.

FIG. 15 is a block diagram illustrating a configuration of an encoded data processing unit.

[Description of Code] 100 Object Extraction Unit 101-103 Object Encoding Unit 104-106 Encoded Data Processing Unit 107 Controller 108 Multiplexing Unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK22 LA00 LB07 MA00 MA05 MB00 PP05 PP06 PP07 PP28 RB18 TA60 TB04 TB18 TC18 TD01 UA02

Claims (4)

[Claims] 1. An apparatus for forming a desired image by multiplexing one or more partial images having an arbitrary shape, wherein said apparatus obtains encoded partial image data obtained by encoding each of said partial images. A data input unit, an encoded data processing unit that reduces the code amount by deleting predetermined encoded data in the input partial image encoded data, and detects a code amount of each partial image encoded data. A code amount controller that controls the coded data processing unit and manages a code allocation amount and a total code amount to each of the partial image coded data forming an image; A data processing device characterized in that the total code amount after multiplexing is reduced to a predetermined code amount or less by deleting some data. 2. The code amount controller extracts a visual feature amount such as a motion, a color, and a size of each partial image, and determines a code amount to be allocated to each partial image based on the visual feature amount. The data processing device according to claim 1, wherein: 3. The coded data is for performing inter-frame predictive coding, and the coded data processing unit reduces a code amount by deleting frames in order from a frame that is not referred to by others. The data processing device according to claim 1, wherein 4. The encoded data has a hierarchical structure in which decoding is performed by using decoding information of a lower layer when decoding an upper layer. 4. The data processing apparatus according to claim 1, wherein the amount of code is reduced by deleting coded data in order from a higher layer.