JP2009182810A - Target code amount calculating apparatus, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a target code amount calculating apparatus calculating a target code amount per an area to be encoded so as to reduce a difference in image quality between image data encoding areas when encoding and transmitting some areas in one frame. <P>SOLUTION: A total target code amount calculating part 2 calculates a total target code amount or a target code amount for the entire of each area to which image data is transmitted. A compressibility calculating part 3 divides the total target code amount by a sum data amount before encoding each area to which the image data is transmitted to calculate a compressibility. An area-based target code amount calculating part 4 multiplies a data amount before encoding by the compressibility per area to which the image data is transmitted to calculate the target code amount per area. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a target code amount calculation device, a target code amount calculation method, and a target code amount calculation program for calculating a target code amount that represents a maximum allowable value of the amount of data after encoding in encoding of image data.

  Various devices for transmitting and receiving image data have been proposed. For example, Patent Document 1 describes a thin client and a server that transmit and receive image data.

  Further, when image data is transmitted / received via a communication network, the image data may be encoded in order to compress the data amount. An image processing apparatus that encodes image data is described in Patent Document 2, for example. The image processing apparatus described in Patent Document 2 performs quantization and data encoding, and counts the amount of encoded data while the encoding process proceeds. If the data amount of the encoded data exceeds the set target code amount in the middle of the encoding process, the quantization step is changed and the encoding process is resumed. Encoding is performed.

JP 2004-228877 A (paragraphs 0036-0041, FIG. 1) Japanese Patent Laying-Open No. 2004-320479 (paragraphs 0044 and 0045, FIG. 3)

  In a system in which image data is transmitted and received between a transmission-side device and a reception-side device, image processing for encoding image data is performed at the transmission-side device so that the amount of encoded data is kept below the target code amount, and then reception is performed. Suppose that it is transmitted to the side device. In this case, it becomes easy to control the encoded data amount to be transmitted and received to be equal to or less than the target code amount.

  Further, as a mode of transmitting image data to another apparatus, a transmission mode of encoding and transmitting image data of a rectangular area including an updated area in the image is conceivable. In this transmission mode, when only a part of the image of the area is updated, it is not necessary to encode and transmit all the image data representing the entire image, and it is only necessary to encode and transmit a part of the image data. The amount of data to be transmitted can be reduced.

  However, when there are a plurality of updated regions in the image and the image data of each region is encoded to the same target code amount or less, the image quality of the encoded data is reduced due to the difference in the area of each updated region. May vary from region to region. For example, assume that two areas A and B are updated in one screen. Since the areas of the regions A and B are not necessarily the same, the image data amounts of the regions A and B are not necessarily the same. If such image data is encoded so as to be equal to or less than the same target code amount, the image quality of the code data may differ between the region A and the region B. If the encoded data in these areas is decoded, the image quality will be different between areas belonging to one image.

  In view of this, the present invention relates to each encoding target region so as to reduce a difference in image quality between regions in which image data is encoded when a part of the region in one frame is encoded and transmitted. It is an object of the present invention to provide a target code amount calculation device, a target code amount calculation method, and a target code amount calculation program that can calculate a target code amount.

  A target code amount calculation apparatus according to the present invention calculates a target code amount when encoding image data in an area within one frame in which image data is transmitted via a communication network. An overall target code amount for calculating an overall target code amount that is a target code amount for the entire region in which image data is transmitted by dividing the communication band of the communication network by a predetermined frame rate. And calculating a compression rate when encoding the image data by dividing the overall target code amount by the total data amount of the image data before encoding in each region to which the image data is transmitted. For each area where image data is transmitted, the compression rate calculation means multiplies the data amount of the image data before encoding in the area by the compression ratio calculated by the compression ratio calculation means. To, characterized in that it comprises a region-specific target code amount calculation means for calculating a target code amount when encoding the image data of the region.

  Further, the target code amount calculation method of the present invention is a target code amount for calculating a target code amount when encoding image data in an area within one frame and where image data is transmitted via a communication network. A calculation method, wherein a communication band of the communication network is divided by a predetermined frame rate to calculate an overall target code amount that is a target code amount for each entire region in which image data is transmitted, By dividing the overall target code amount by the total amount of image data before encoding of each area in which image data is transmitted, the compression rate when encoding the image data is calculated, and the image data is transmitted For each individual area, the target code amount when encoding the image data of the area is obtained by multiplying the data amount of the image data before the encoding of the area by the calculated compression rate. Characterized in that it out.

  The target code amount calculation program of the present invention is installed in a computer that calculates a target code amount when encoding image data in an area within one frame and where image data is transmitted via a communication network. The target code amount calculation program is a target code amount for the entire area in which image data is transmitted by dividing the communication bandwidth of the communication network by a predetermined frame rate to the computer. Overall target code amount calculation processing for calculating the overall target code amount, dividing the overall target code amount by the total data amount of the image data before encoding in each of the areas to which the image data is transmitted. Compression rate calculation processing for calculating the compression rate when encoding, and encoding of each region for each region where image data is transmitted Multiplying the data amount of the image data by the compression rate calculated in the compression rate calculation processing to execute a target code amount calculation processing for each region that calculates a target code amount when encoding the image data of the region It is characterized by that.

  According to the present invention, when encoding and transmitting a partial area in one frame, each encoding target area is reduced so as to reduce a difference in image quality between areas in which image data is encoded. It is possible to calculate the target code amount.

  Embodiments of the present invention will be described below with reference to the drawings.

Embodiment 1. FIG.
The target code amount calculation apparatus of the present invention calculates a target code amount that represents the maximum allowable amount of data after encoding in the process of encoding image data. The data encoded according to the target code amount is transmitted to the image data receiving device via the communication network. The target code amount calculation apparatus according to the present invention calculates a target code amount for each region when encoding image data in a region within one frame and where image data is transmitted via a communication network. calculate. In addition, the area within one frame and where image data is transmitted via the communication network is specifically an area including an area in which an image is updated in one frame. FIG. 1 is a schematic diagram illustrating an example of an area where image data is encoded and transmitted. It is assumed that the image is updated in the regions 82 and 83 indicated by the broken lines among the regions in the frame 81, and the image is not updated in the regions other than the regions 82 and 83. In this case, the image data in the areas 82 and 83 is encoded and transmitted to the receiving side apparatus. In this example, the target code amount calculation apparatus of the present invention calculates the target code amount individually for each of the regions 82 and 83. The image data in the areas 82 and 83 is encoded and transmitted according to the target code amount for each area.

  In the example shown in FIG. 1, two areas in which image data is encoded and transmitted are shown. However, the number of areas in which image data is encoded and transmitted in one frame is not particularly limited. There may be one or three or more.

  FIG. 2 is a block diagram showing the first embodiment of the target code amount calculation apparatus of the present invention. The target code amount calculation apparatus 1 of the present invention includes an overall target code amount calculation unit (overall target code amount calculation unit) 2, a compression rate calculation unit (compression rate calculation unit) 3, and a target code amount calculation unit for each region (region A separate target code amount calculation means) 4.

  The overall target code amount calculation unit 2 calculates the overall target code amount based on a communication band of a communication network to which encoded image data is transmitted and a predetermined frame rate. The overall target code amount is a target code amount for each entire region in which image data is transmitted. In the example illustrated in FIG. 1, the overall target code amount is the maximum allowable amount of data after encoding (upper limit value) when encoding the entire image data in each of the regions 81 and 82 to which image data is transmitted. ). The communication band of the communication network may be input from the outside, for example. The frame rate is the frame rate of the image data receiving device, and may be stored in advance in a storage device (not shown) included in the target code amount calculation device 1, for example. The overall target code amount calculation unit 2 calculates the overall target code amount by dividing the communication band of the communication network by the frame rate.

  The compression rate calculation unit 3 calculates the compression rate when encoding the image data of each region to which the image data is transmitted. The compression rate calculation unit 3 divides the overall target code amount calculated by the overall target code amount calculation unit 2 by the total data amount of the image data before encoding in each region to which the image data is transmitted, and thereby calculates the compression rate. Ask. For example, in the example shown in FIG. 1, the compression rate is calculated by dividing the overall target code amount by the sum of the image data amounts before encoding of the regions 81 and 82. In the present embodiment, the compression rate calculated by the compression rate calculation unit 3 is a common compression rate when the image data of each region is encoded for each region.

  Note that the amount of data before encoding in each region to which image data is transmitted may be designated from the outside. Alternatively, the target code amount calculation device itself identifies individual regions (that is, regions in which an image is rewritten within one frame) encoded with image data, and the amount of image data in each region and The total amount of data may be calculated.

  The target code amount calculation unit 4 for each region calculates a target code amount for encoding the image data of each region for each region to which the image data is transmitted. The region-specific target code amount calculation unit 4 selects each region to which image data is transmitted, and multiplies the data amount of the image data before encoding in that region by the compression rate calculated by the compression rate calculation unit 3. Thus, the target code amount for each region is calculated.

  The overall target code amount calculation unit 2, the compression rate calculation unit 3, and the region-specific target code amount calculation unit 4 may be realized by, for example, a CPU that operates according to a program (target code amount calculation program).

  Next, the operation will be described. FIG. 3 is a flowchart showing an example of processing progress of the target code amount calculation apparatus of the present invention. First, the overall target code amount calculation unit 2 calculates the overall target code amount by dividing the communication band of the communication network by the frame rate on the receiving device side (step S61). As already described, the communication band of the communication network may be input from the outside, for example. The frame rate may be stored in advance in a storage device (not shown) provided in the target code amount calculation device 1, for example.

  Next, the compression rate calculation unit 3 divides the overall target code amount calculated in step S61 by the total data amount of the image data before encoding in each region to which the image data is transmitted, thereby performing encoding. The compression ratio is calculated (step S62).

  Next, the region-specific target code amount calculation unit 4 selects each region to which image data is transmitted, and multiplies the data amount of the image data before encoding in that region by the compression rate obtained in step S62. Thus, the target code amount for each area is calculated (step S63).

  Using the target code amount for each region obtained by the above operation, the image data of each region may be sequentially encoded below the target code amount. This process may be performed by the target code amount calculation apparatus itself or by another apparatus.

  Next, the operation of the present invention will be described with a specific example. In the example shown below, it is assumed that the communication bandwidth in the communication network is 3000 kbps and the predetermined frame rate is 30 fps. FIG. 4 is a schematic diagram showing an example of an area where image data is encoded and transmitted. Among the areas in the frame 90, the image is rewritten in three rectangular areas 91 to 93 indicated by broken lines. It is assumed that the image data of the rectangular areas 91 to 93 is encoded and transmitted. Hereinafter, the rectangular areas 91, 92, and 93 are referred to as a first area, a second area, and a third area, respectively. Note that the numerical values shown in parentheses in FIG. 4 are the coordinates of the pixel at the apex of the rectangular area.

  First, the overall target code amount calculation unit 2 calculates the overall target code amount by dividing the communication bandwidth of 3000 kbps of the communication network by the frame rate of 30 fps (step S61). In this example, the overall target code amount is 3000 kbps / 30 fps = 100 kbit. The overall target code amount 100 kbit is “when encoding the entire image data of the first area 91, the second area 92, and the third area 93, the encoded data quantity does not exceed 100 kbit. ".

  Next, the compression rate calculation unit 3 uses the overall target code amount 100 kbit calculated in step S61 as the image data before encoding the first area 91, the second area 92, and the third area 93 to which the image data is transmitted. Is divided by the total data amount to calculate the compression rate (step S62).

  For example, the x coordinate range of pixels belonging to the first area 91 is 100 to 299, the y coordinate range is 100 to 299, and the number of pixels belonging to the first area 91 is 200 × 200 = 40000. . In addition, the x coordinate range of pixels belonging to the second area 92 is 400 to 799, the y coordinate range is 200 to 599, and the number of pixels belonging to the second area 92 is 400 × 400 = 16000. . Furthermore, the x coordinate range of the pixels belonging to the third region 93 is 900 to 1023, the y coordinate range is 700 to 767, and the number of pixels belonging to the third region 93 is 124 × 68 = 8432. . Further, it is assumed that the data amount per pixel before encoding is 24 bits. In this case, the total data amount of the three rectangular areas 91 to 93 before encoding is 24 × (40000 + 16000 + 8432) = 5006868 bits, that is, about 5002 kbit. The compression rate calculation unit 3 divides the overall target code amount 100 kbit by the data amount 5002 kbit to obtain a compression rate 100/5002. 100/5002 is about 1/50, and hereinafter, the obtained compression ratio will be described as 1/50 for convenience.

  After calculating the compression rate, the target code amount calculation unit 4 for each region selects each of the rectangular regions 91 to 93, and calculates the target code amount for each region (step S63). For example, since the number of pixels in the first area 91 is 40000 and the data amount per pixel is 24 bits, the data amount in the first area 91 is 24 bits × 40000 = 960 kbits. The region-specific target code amount calculation unit 4 multiplies the data amount by a compression ratio of 1/50 to calculate a target code amount at the time of encoding the first region 91. Therefore, the target code amount of the first area 91 is 19.2 kbit. Similarly, since the number of pixels in the second area 92 is 160000, the data amount is 24 bits × 160000 = 3840 kbits. The region-specific target code amount calculation unit 4 multiplies the data amount by the compression ratio 1/50 to calculate the target code amount at the time of encoding the second region 92. Therefore, the target code amount of the second area 92 is 76.8 kbit. Similarly, since the number of pixels in the third region 93 is 8432, the data amount is 24 bits × 8432 = 202 kbit (202368 bits). The region-specific target code amount calculation unit 4 multiplies the data amount by the compression ratio 1/50 to calculate the target code amount at the time of encoding the third region 93. Therefore, the target code amount of the third area 93 is 4 kbit.

  Therefore, when the image data 960 kbit of the first area 91 is encoded, it is only necessary that the encoded data amount is 19.2 kbit or less. Even when the image data of the other rectangular areas 92 and 93 is encoded, the encoding may be performed so that the encoded data amount is equal to or less than the target code amount of the rectangular area.

  According to the present invention, the compression rate calculation unit 3 calculates the compression rate by dividing the overall target code amount by the total data amount of the image data before encoding in each region. Then, the target code amount calculation unit 4 for each region calculates a target code amount corresponding to each region by multiplying the data amount before region encoding by the compression rate for each region. Therefore, since the target code amount for each area is calculated with a common compression rate, the target of each area is set so that the difference in image quality of the image data after encoding between areas in the same frame is reduced. The code amount can be obtained.

Embodiment 2. FIG.
The target code amount calculation apparatus according to the first embodiment determines the target code amount by sharing the compression rate of image data between regions. Even if the image data of the region is encoded such that the encoded data amount is equal to or less than the target code amount, the encoded data amount may be larger than the target code amount. In this case, in order to suppress the data amount when all the regions are encoded to be equal to or less than the overall target code amount, it is necessary to increase the compression rate in other regions. In addition, as a result of encoding, the amount of data after encoding in a certain region may be significantly less than the target code amount in that region. In such a case, it is possible to reduce the compression rate of an area that has not yet been encoded and improve the image quality of that area.

  The target code amount calculation apparatus according to the second embodiment recalculates the compression rate for each area where the encoding of the image data is not completed when there is an area where the encoding of the image data is completed. Based on the compression rate, the target code amount of each area where the encoding of the image data is not completed is recalculated.

  FIG. 5 is a block diagram showing a second embodiment of the target code amount calculation apparatus of the present invention. The target code amount calculation apparatus of the present embodiment includes an overall target code amount calculation unit 2, a compression rate calculation unit 3, a region-specific target code amount calculation unit 4, and a subtraction unit (subtraction unit) 5. Constituent elements similar to those of the first embodiment are denoted by the same reference numerals as those in FIG. 2, and detailed description thereof is omitted. However, in addition to the processing described in the first embodiment, the compression rate calculation unit 3 and the region-specific target code amount calculation unit 4 have completed encoding of image data in each region to which image data is transmitted. If there is a region (hereinafter referred to as an encoding completion region), the processing described later is also executed.

  When there is an encoding completion region, the subtraction unit 5 subtracts the total amount of image data after encoding in the encoding completion region from the overall target code amount. The result of subtracting the total amount of image data after encoding in the encoding completion region from the total target code amount means the target code amount for each entire region that has not yet been encoded.

  For example, in the example shown in FIG. 4, it is assumed that the overall target code amount (P) is calculated for each of the rectangular regions 91 to 93 and the target code amount of each rectangular region 91 to 93 is calculated. Furthermore, after that, when the image data of the first area 93 is encoded, it is assumed that the encoded data amount is D1. In this case, the subtracting unit 5 subtracts the encoded data amount D1 of the first area 91, which is the encoding completion area, from the overall target code quantity P obtained first to obtain P-D1. This P-D1 is the target code amount for the second region 92 and the third region 93 that are not encoded, and the encoded data amount of the second region 92 and the third region 93 is equal to or less than P-D1. If there is, the data amount after the encoding of the three rectangular areas 91 to 93 is suppressed to P or less.

  Similarly, it is assumed that not only the first area but also the image data of the second area has been encoded, and the encoded data amount in the second area is D2. In this case, the subtracting unit 5 subtracts the total amount (D1 + D2) of the encoded image data of the first region 91 and the second region 92, which are the encoding completion regions, from the overall target code amount P obtained first. P- (D1 + D2) is obtained. This P− (D1 + D2) is the target code amount for the third region 93 that is not encoded.

  In this way, the subtraction unit 5 first calculates the overall target every time the encoding process is completed in one area among the areas where the image data is transmitted and the encoding completion area is increased by one. A process of subtracting the total amount of image data after encoding in the encoding completion area from the code amount is performed.

  When calculating the compression rate for the first time, the compression rate calculation unit 3 in the second embodiment uses the overall target code amount derived by the overall target code amount calculation unit 2 in the same manner as in the first embodiment. Is calculated. In addition, the compression rate calculation unit 3 performs the subtraction when the encoding process is sequentially performed for each area and there is an area where the image data is not yet encoded among the areas to which the image data is transmitted. The subsequent overall target code amount is divided by the total amount of image data before encoding in each region where encoding is not performed, and the compression rate is recalculated. That is, the result of subtracting the encoded image data amount of each encoding completion region from the initially obtained overall target code amount is divided by the total amount of image data of each region that has not been encoded, and a new Calculate the compression ratio.

  When calculating the target code amount for each region for the first time, the region-specific target code amount calculation unit 4 in the second embodiment calculates the target code amount in the same manner as in the first embodiment. Thereafter, encoding is performed in a region where image data is transmitted, and when the compression rate calculation unit 3 recalculates the compression rate, the target code amount in a region where encoding is not performed using the compression rate. Is recalculated for each region. The area-specific target code amount calculation unit 4 multiplies the data amount of the image data before encoding of the area by the new compression rate for each area where the image data is not yet encoded. Recalculate the target code amount of the region.

  The subtracting unit 5 is realized by, for example, a CPU that operates as the overall target code amount calculating unit 2, the compression rate calculating unit 3, and the region-specific target code amount calculating unit 4, and the CPU performs processing as the subtracting unit 5 according to a program. May be.

Next, the operation will be described.
The operation until the target code amount for each region is calculated for the first time is the same as that in the first embodiment, and the processes in steps S61 to S63 are performed. Thereafter, every time encoding of one region is completed, the target code amount recalculation process shown below is repeated. However, the target code amount recalculation process does not have to be performed when encoding is completed for all the areas in which image data is transmitted.

  FIG. 6 is a flowchart illustrating an example of target code amount recalculation processing. When encoding of one region is completed, the subtracting unit 5 subtracts the total amount of image data after encoding in the encoding completion region from the overall target code amount obtained first (step S65).

  Next, the compression rate calculation unit 3 recalculates the compression rate by dividing the subtraction result in step S65 by the total amount of image data before encoding in each region that has not been encoded (step S66). ).

  Subsequently, the target code amount calculation unit 4 for each region performs encoding by multiplying the image data amount in the region by a new compression rate for each region where the image data is not yet encoded. The target code amount of each area that has not been calculated is recalculated (step S67).

  Next, the operation of this embodiment will be described with a specific example. The example shown below will be described with reference to FIG. 4 as in the specific example illustrated in the first embodiment. Similar to the specific example described in the first embodiment, the overall target code amount is calculated as 100 kbits in step S61, and the processes of steps S62 and S63 are performed to calculate the target code amounts of the rectangular areas 61 to 63. And After that, the image data in the first area 91 is encoded to be equal to or less than the target code amount (19.2 kbit), and as a result, the encoded data amount in the first area 91 is 16 kbit.

  At this time, the subtracting unit 5 subtracts the total amount of image data after encoding in the encoding completion area from the overall target code amount (100 kbit) obtained first. In this example, since only the first area 91 is encoded, 16 kbit is subtracted from 100 kbit, and 84 kbit is obtained as a subtraction result (step S65).

  Next, the compression rate calculation unit 3 divides the subtraction result (84 kbit) by the total amount of image data before encoding in the second region 92 and the third region 93 that have not been encoded, and calculates the compression rate. Recalculate. As described in the specific example of the first embodiment, assuming that the number of pixels in the second region 92 and the third region 93 is 160000 and 8432, respectively, and the data amount of one pixel is 24 bits, The total amount of data before encoding in 92 and the third area 93 is 24 × (160000 + 8432) = 4042368 bits = 4042.368 kbits. Therefore, the compression rate calculation unit 3 calculates 84 / 4042.368 and obtains a new compression rate (step S66). Since 84 / 4042.368 = about 1/48, the following description will be made assuming that the new compression rate is 1/48 for convenience.

  Next, the region-specific target code amount calculation unit 4 calculates the target code amounts of the second region 92 and the third region 93 that have not been encoded (step S67). The region-specific target code amount calculation unit 4 multiplies the data amount (3840 kbit) of the second region 92 by 1/48 to set the new target code amount to 80 kbit. Similarly, the region-specific target code amount calculation unit 4 multiplies the data amount (202 kbit) of the third region 93 by 1/48 to obtain a new target code amount of 4.2 kbit.

  Further, the operations of steps S65 to S66 may be performed when the second region 92 and the like are further encoded.

  According to this embodiment, every time encoding of one region is completed, the compression rate is recalculated, and the target code amount of the region that has not been encoded is recalculated according to the new compression rate. It is possible to increase the accuracy of code amount control when encoding each region. For example, it is assumed that an image in a certain area is very simple and can be compressed at a compression rate higher than the compression rate calculated by the compression rate calculation unit 3 as a result of encoding. Then, the compression rate of the other area can be lowered, and the image quality after encoding of the other area can be improved. In addition, for example, it is assumed that encoding cannot be performed below the target code amount as a result of encoding in a certain area. In this case, it is possible to control so as to increase the compression rate of other regions.

  In the second embodiment, there are cases where the compression rates of the respective regions differ from each other, but the difference in image quality after encoding between the respective regions is smaller than when the target code amount of each region is uniformly determined. Therefore, in the second embodiment, it is possible to increase the accuracy of code amount control when encoding individual areas while reducing the difference in image quality after encoding between the areas.

  Hereinafter, modifications of the first embodiment and the second embodiment will be described.

  In the first embodiment and the second embodiment, the region-specific target code amount calculation unit 4 has a data amount of image data before encoding in a region where image data is not encoded being equal to or less than a predetermined threshold. If there is, the target code amount of the area may be set to the maximum value that the target code amount can take.

  The maximum value that the target code amount can take is a value that is determined as the upper limit value of the target code amount in an apparatus that encodes image data. In a device that encodes image data, if the target code amount can be set to infinity, the maximum value that the target code amount can take is infinite. If the target code amount is set to a large value (maximum value) in this way, when encoding is performed for an area in which image data is transmitted, the compression rate is lowered, and the image quality for that area is improved.

  Further, it can be said that the area where the data amount of the image data before encoding is equal to or smaller than the predetermined threshold is an area whose area is equal to or smaller than the predetermined value. In such a region with a small amount of data, even if the compression is not performed at a high compression rate, the influence on the data amount after encoding when the entire region is encoded is small. That is, the increase in the amount of code when the entire area is encoded does not become significant. In addition, in the frame, such an image update with a very small area frequently occurs in the vicinity of an object such as a cursor, an icon, or the like where definition is important. In addition, when the definition of such an object is greatly reduced, operability is often hindered even if the frame rate can be maintained. By setting the target code amount to the maximum value in a region where the data amount of image data before encoding is equal to or less than a predetermined threshold, the image quality after encoding of such a small area region can be maintained above a certain level. .

  In each of the above embodiments, the target code amount calculation apparatus itself may determine an area where an image has been rewritten within one frame and specify an area where image data is transmitted. FIG. 7 is a block diagram illustrating a configuration example in the case where the target code amount calculation apparatus includes an area specifying unit (area specifying unit) 6 that specifies an area in which image data is transmitted in one frame. Constituent elements similar to those in the above embodiments are given the same reference numerals as those in FIGS. 2 and 5 and description thereof is omitted.

  The area specifying unit 6 specifies a rectangular area including an area in which an image is updated in one frame as an area to which image data is transmitted. An example of how the area specifying unit 6 specifies such an area will be described.

  FIG. 8 is an explanatory diagram illustrating an example of an area in which an image is transmitted within one frame. In FIG. 8, a hatched portion 96 is a portion where the image is rewritten in the frame 95, and a white portion is a portion where the image is not rewritten. The area specifying unit 6 specifies the rectangular area 97 including the portion 96 where the image is updated as an area for encoding and transmitting image data.

  Here, in the frame, the x-axis extends in the right direction, and the right-side pixel has a larger coordinate value of the x-coordinate. The y-axis extends downward, and the lower pixel has a larger y-coordinate value.

  The area specifying unit 6 sequentially selects from the first line to the last line, and determines whether or not there is a rewritten pixel in the selected line. Hereinafter, for convenience, a row in which the rewritten pixel exists is referred to as an updated row, and a row in which the rewritten pixel does not exist is referred to as a non-updated row. When the immediately preceding line is a non-updated line and the selected line is an updated line, the area specifying unit 6 sets that line as the head of the updated line. When the immediately preceding line is an updated line and the selected line is a non-updated line, the immediately preceding line is set as the end of the updated line. However, in such a case, if there are a small number of non-updated rows between successive update rows, there is a possibility that the number of regions cut out from the frame as the rectangular region 97 becomes too large. Accordingly, when the area specifying unit 6 switches from the update line to the non-update line, the region specifying unit 6 continues the process of selecting another line and determining whether or not it is an update line, and after changing from the update line to the non-update line, May be determined to be the end of the update row when the condition that the number of is continuous for a predetermined number or more is satisfied. After specifying the end of the update line, the area specifying unit 6 repeats the process of specifying the start and end of the update line again when switching from the non-update line to the update line. FIG. 9 is an explanatory diagram illustrating an example of the beginning and the end of an update line. The area specifying unit 6 specifies the beginning and end of the update line as illustrated in FIG.

  Subsequently, the area specifying unit 6 performs the update from the first column (the leftmost column in this example) to the last column (the rightmost in this example) for each group of consecutive update rows from the beginning to the end of the update row. Are sequentially selected, and it is determined whether or not there is a rewritten pixel in the selected column. Hereinafter, for convenience, a column in which the rewritten pixel exists is referred to as an updated column, and a column in which the rewritten pixel does not exist is referred to as a non-updated column. When the immediately preceding column is a non-update column and the selected column is an update column, the area specifying unit 6 sets the column as the left end of the update column. Further, when the immediately preceding column is an updated column and the selected column is a non-updated column, the region specifying unit 6 sets the immediately preceding updated column as the right end of the updated column. However, in such a case, if there are a small number of non-update columns between successive update columns, the number of regions cut out from the frame as the rectangular region 97 may be too large. Therefore, when the region specifying unit 6 switches from the update column to the non-update column, the region specifying unit 6 further selects a column and determines whether or not the column is an update column. When the condition that a predetermined number of columns are continuous is satisfied, the updated column may be determined as the right end of the updated column. After specifying the right end of the update column, the area specifying unit 6 repeats the process of specifying the left end and the right end of the update column again when the non-update column is switched to the update column again. FIG. 10 is an explanatory diagram showing an example of the left end and the right end of the update column, and the vertical lines shown by broken lines in FIG. 10 represent the left end and the right end of the update column. The area specifying unit 6 specifies the left end and the right end of the update sequence as illustrated in FIG.

  Next, the area specifying unit 6 sequentially selects the range surrounded by the beginning and end of the update row and the left end and right end of the update column one by one. In the example shown in FIG. 10, the three ranges are selected one by one sequentially.

  The area specifying unit 6 sequentially extracts each column in the selected range, and specifies the upper end and the lower end of the update pixel (updated pixel) in each column. Further, the region specifying unit 6 specifies the y coordinate of the uppermost pixel among the upper ends of the update pixels in each column and the y coordinate of the lowermost pixel among the lower ends of the update pixels in each column. The region specifying unit 6 converts the image data into a rectangular region in a range from the y coordinate of the uppermost pixel to the y coordinate of the lowermost pixel and a range from the left end x coordinate to the right end x coordinate of the update sequence. Identifies the area to be transmitted.

  FIG. 11 is an explanatory diagram illustrating an example of a range surrounded by the beginning and end of the update row and the left end and right end of the update column. For example, when the range illustrated in FIG. 11 is selected, the region specifying unit 6 sequentially extracts each column such as the columns 101 and 111 illustrated in FIG. 11 and specifies the upper end and the lower end of the update pixel in each column. For example, when the column 101 is extracted, the pixel 102 is specified as the upper end of the update pixel, and the pixel 103 is specified as the lower end of the update pixel. Then, the region specifying unit 6 specifies the y coordinate of the uppermost pixel (pixel 112 in this example) among the upper ends 102 and 112 of the update pixels in each column. Further, the y coordinate of the lowermost pixel among the lower ends 103, 113, etc., of the update pixel in each column is specified. Then, a range from the y-coordinate of the pixel 112 to the y-coordinate of the pixel 113 and a rectangular region from the left end x coordinate to the right end x coordinate of the update sequence are specified as regions to which image data is transmitted.

  After specifying each region to which image data is transmitted by the above operation, the target code amount for each region may be calculated by the operation described in the first embodiment, the second embodiment, or the like.

  The region specifying method described here is an exemplification, and the region specifying method is not limited to the above method. The area specifying unit 6 sequentially takes out a rectangular area having a constant height and width from one frame (for example, a rectangular area having 64 pixels in both vertical and horizontal directions), and if an update pixel is included in the rectangular area The rectangular area may be specified as an area to which image data is transmitted.

  The area specifying unit 6 is realized, for example, by a CPU that operates as the overall target code amount calculating unit 2, the compression rate calculating unit 3, and the area-specific target code amount calculating unit 4, and the CPU performs processing as the area specifying unit 6 according to a program. May be performed.

  In each of the above embodiments, the target code amount calculation device itself may encode the image data in each region. FIG. 12 is a block diagram illustrating a configuration example in a case where the target code amount calculation apparatus includes a region encoding unit (region encoding unit) 7 that encodes image data for each region. Constituent elements similar to those in each of the above embodiments are given the same reference numerals as those in FIGS. 2, 5, and 7, and description thereof is omitted. In FIG. 12, the area specifying unit 6 is not shown, but the above-described area specifying unit 6 may be provided.

  The region encoding unit 7 performs a process of encoding the image data to a code amount equal to or less than the target code amount determined by the region-specific target code amount calculation unit 4 for each region where the image data is transmitted. FIG. 13 is a block diagram illustrating an example of the region encoding unit 7. In the following, a case where the region encoding unit 7 converts image data into a Huffman code will be described as an example. The region encoding unit 7 includes a wavelet transform unit 12, an intermediate code generation unit 13, a first determination unit 14, a second determination unit 15, a requantization unit 16, and an encoding unit 17. .

  The wavelet transform unit 12 performs a process for wavelet transforming the image data up to a predetermined number of decomposition levels for each region to which the image data in one frame is transmitted. This “predetermined number of decomposition levels” is denoted as “N”. In the drawings, the number of decomposition levels may be written as Rlevel. FIG. 14 is an explanatory diagram schematically showing the wavelet transform. The wavelet transform unit 12 performs wavelet transform on the area image data 100. The types of coefficients obtained by one wavelet transform include LL component, LH component, HL component, and HH component, and the wavelet transform unit 12 recursively performs wavelet transform on the LL component. Hereinafter, the LH component, the HL component, and the HH component are collectively referred to as xx. Further, the LL component and the xx component when the number of decomposition levels is k are referred to as “kLL component” and “kxx component”, respectively. FIG. 14 shows an xx component (1xx) having a decomposition level number 1, an LL component (2LL) and an xx component (2xx) having a decomposition level number 2. 14 illustrates wavelet transformation up to the number of decomposition levels “2”. However, the number of decomposition levels when the wavelet transform unit 12 performs wavelet transformation is not limited to 2, and the number of decomposition levels equal to or greater than 3. Wavelet transformation may be performed up to. In the following description, an example will be described in which wavelet transformation is performed so that coefficients after wavelet transformation (wavelet coefficients) are expressed in YUV format and have Y, U, and V values.

  The intermediate code generation unit 13 is a value obtained by quantizing the coefficient obtained by the wavelet transform in a predetermined quantization step or the coefficient itself obtained by the wavelet transform (hereinafter referred to as intermediate code). .) Is generated. Here, a case where the result obtained by quantizing the coefficient obtained by the wavelet transform is an intermediate code will be described as an example. Therefore, in this example, the intermediate code generation unit 13 quantizes the values of Y, U, and V included in the wavelet coefficients in a predetermined quantization step, and sets the quantization result as an intermediate code. Hereinafter, the quantization step is referred to as Q step.

  The intermediate code generation process by the intermediate code generation unit 13 will be described in more detail. Predetermined Q steps and dead zones at the time of intermediate code generation are determined for each intermediate code generation target component such as a 1xx component and a 2xx component. The intermediate code generation unit 13 quantizes the value of the image data whose absolute value is equal to or less than a predetermined dead zone to 0. For example, if the absolute value of a certain Y included in the 1xx component is equal to or less than the dead zone, the Y is quantized to 0. Further, the intermediate code generation unit 13 divides a range in which the absolute value exceeds a predetermined dead zone every predetermined Q steps. Then, the values of the image data belonging to each of these ranges are quantized to the median value of the ranges. For example, it is assumed that the dead zone and the Q step are both 2. In this case, a range in which the absolute value exceeds 2 (that is, a range of less than −2 and a range of greater than 2) is divided into two Q steps, “−4 to −2”, “2 to 4”, “ Define a range such as 4-6 ″. If a certain Y included in the 1xx component belongs to a range such as “2 to 4”, the value of Y is quantized to the median value of the range. Here, Y included in the 1xx component is illustrated, but U and V quantization is performed in the same manner. The same applies to the quantization processing for components other than the 1xx component.

  Further, the intermediate code generation unit 13 may include the result of the run length analysis in the intermediate code. That is, the result of the run length analysis may also be an intermediate code (intermediate data). The run length analysis is a process of counting the number of consecutive data when the same data is continuously arranged. As an aspect of the run length analysis, there is an aspect in which the number of consecutive pixels is counted when pixels whose Y, U, and V values after quantization are all 0 are consecutive. Alternatively, when pixels having the same combination of Y, U, and V after quantization are continuous, the number of consecutive pixels may be counted. FIG. 15 is an explanatory diagram schematically showing an intermediate code generated from wavelet coefficients. In FIG. 15, the continuous number counted by the run length analysis is expressed as “len”. For example, the intermediate code generation unit 13 quantizes Y, U, and V included in the 2LL component, the 2xx component, and the 1xx component as described above, and generates a 2LL intermediate code, a 2xx intermediate code, a 1xx intermediate code, and the like. Alternatively, the run length analysis may be performed on the quantized Y, U, and V, and the result (the counted number of consecutive “len”) may be included in the intermediate code. Hereinafter, a case where len is included in the intermediate code will be described as an example.

Also, the decomposition level number defining the highest quality of the image obtained from the coded data (a m _i.) Is predetermined. Here, _mi is an integer of 1 or more and N or less. The intermediate code generation unit 13 generates an intermediate code for the LL component having the decomposition level number m _{i to} N and the xx component having the decomposition level number 1 to N. In other words, an intermediate code is generated for (m _i ) LL component, (m _i +1) LL component,..., NLL component, 1xx component, 2xx component,.

  The first determination unit 14 performs a process of determining whether or not to encode the xx component (that is, the component other than the LL component) in the intermediate data of the number of decomposition levels in a predetermined range.

The first determination section 14 to make this determination, (xx components of the LL component of the decomposition level number m _i to N, and the division level number 1 to N) generates subject to the above respective components of the intermediate code for each In addition, the frequency (number of occurrences) of intermediate codes in each range used for quantization is counted. Specifically, the first determination unit 14 counts the occurrence frequency of data quantized in a range where the absolute value is equal to or less than a predetermined dead zone for each of Y, U, and V. Further, the frequency of occurrence of data quantized for each range obtained by dividing the range in which the absolute value exceeds the dead zone for each predetermined Q step is counted for each of Y, U, and V.

  FIG. 16 is an explanatory diagram illustrating an example of information (hereinafter referred to as a frequency table) indicating the result of counting the frequency of occurrence of quantized data in each range. FIG. 16 shows an example of the frequency table of Y and U of the 2LL intermediate code. The left column of each frequency table shown in FIG. 16 shows the quantized result, and the right column shows the frequency of occurrence of data quantized to that value (Y or U in the example of FIG. 16). Yes. That is, in the example of Y of the 2LL intermediate code illustrated in FIG. 16, there are 230 pieces of Y data whose absolute value belongs to the range below the dead zone and is quantized to 0, and belongs to the range whose central value is 255. Indicates that there are 44 Y data quantized.

  The first determination unit 14 generates a Huffman table from each frequency table. FIG. 17 is an explanatory diagram illustrating an example of a Huffman table. FIG. 17 illustrates a Huffman table for Y of the 2LL intermediate code. The first determination unit 14 generates a Huffman table as illustrated in FIG. 17 from the Y frequency table of the 2LL intermediate code. Similarly, Huffman tables are also generated from other frequency tables. As shown in FIG. 17, the Huffman table includes each quantized value (0 to 255 in the example shown in FIG. 17) and a code converted from the value. The first determination unit 14 determines the code so that the quantized value and the code have a one-to-one correspondence and the bit frequency of the code is shortened as the frequency of occurrence increases. To generate a Huffman table. By generating the Huffman code, the bit length of the Huffman code for each quantized data value (0 to 255 in the example shown in FIG. 16) is determined. For each Huffman code, the first determination unit 14 obtains the product of the bit length of the Huffman code and the corresponding occurrence frequency, and calculates the sum. If the len is included in the intermediate code, the bit length when each len is encoded (for example, the bit length when gamma-coded) is also added to the above sum. The 1st determination part 14 calculates | requires these sum regarding each xx component from a 1xx component to a (N-1) xx component, and a (N-1) LL component. When the total is larger than the threshold value that is the target code amount of the code amount, the first determination unit 14 selects the number of decomposition levels in a predetermined range in descending order, and the intermediate data of the selected number of decomposition levels A data amount when the xx component is encoded is estimated, and whether or not the xx component of the intermediate data is encoded is determined according to the estimated data amount. The operation flow of this determination will be described later with reference to FIGS. 19 and 20.

The second determination unit 15 determines whether or not to re-quantize the xx component (component other than the LL component) of the intermediate code of each decomposition level number that is smaller than the number of decomposition levels in the predetermined range. Judging. Let m _{q be} the maximum value of the number of decomposition levels for which it is determined whether or not to perform requantization. In this case, the number of decomposition levels in the above defined range is the number of decomposition levels in the range of m _q +1 to N.

When the second determination unit 15 determines that requantization is performed, the requantization unit 16 determines the number of decomposition levels (1 to 1) smaller than the number of decomposition levels in a predetermined range (that is, m _q +1 to N). The xx component of the intermediate code of m _q ) is requantized.

  The encoding unit 17 encodes the value requantized from the intermediate code by the intermediate code and the requantization unit 16 into a Huffman code. Further, when len (continuous number counted by continuous length analysis) is included in the intermediate code, the len is also encoded. In this case, the encoding part 17 should just encode len to a gamma code, for example. If the bit length when len is expressed in binary number is l, the encoding unit 17 converts len to gamma code by adding “l−1” 0s to the head of the binary bit. To do. The encoding unit 17 converts the intermediate code or the result of requantization into a Huffman code, and converts len into, for example, a gamma code, whereby image data is encoded.

  FIG. 18 is an explanatory diagram schematically illustrating an example of the result of Huffman coding. FIG. 18 illustrates a case where encoding is performed on the 2LL component, the 2xx component, and the 1xx component. When the data before encoding includes len as well as Y, U, and V, the encoded data of Y, U, V, and len are included in the encoding result of the 2LL component or the like.

  A region encoding unit 7 including a wavelet transform unit 12, an intermediate code generation unit 13, a first determination unit 14, a second determination unit 15, a requantization unit 16, and an encoding unit 17, For example, it may be realized by a CPU that operates as the overall target code amount calculation unit 2, the compression rate calculation unit 3, and the region-specific target code amount calculation unit 4, and the CPU may perform processing as the region encoding unit 7 according to a program. .

  Next, the operation of the region encoding unit 7 will be described. FIG. 19 and FIG. 20 are flowcharts showing an example of processing progress of the region encoding unit 7. Note that “output” described in the flowchart represents “encoding processing”.

  First, the wavelet transform unit 12 performs wavelet transform on the image data in the area where the image is transmitted up to the decomposition level number N.

Then, the intermediate code generation unit 13 generates an intermediate code with respect to the LL component having the decomposition level number m _{i to} N and the xx component having the decomposition level number 1 to N. That is, the intermediate code generation unit 13 quantizes Y, U, and V included in these components according to the dead zone and Q step determined according to the type of component. As already described, the intermediate code generation unit 13 quantizes Y, U, and V whose absolute values are equal to or less than a predetermined dead zone to 0. Further, a range where the absolute value exceeds a predetermined dead zone is divided every Q steps, and Y, U, and V belonging to each range are quantized to a median value of the range. Further, for example, when pixels having the same combination of Y, U, and V are consecutive, the number of consecutive (len) is counted. The intermediate code generation unit 13 stores the Y, U, and V quantization results and len as intermediate codes in a storage device (not shown) included in the target code amount calculation device.

Further, the first determination unit 14 targets each of Y, U, and V for the intermediate code of the decomposition level number m _{i to} N that generated the intermediate code and the intermediate code of the decomposition level number 1 to N of the xx component. A frequency table is generated and stored in a storage device (not shown) included in the target code amount calculation device. The first determination unit 14 may generate a frequency table by counting the frequency of occurrence in each range determined by the dead zone and the Q step for each of Y, U, and V included in each component (the above step). S1).

Next, the first determination unit 14 substitutes m _i (the number of decomposition levels that defines the highest image quality of the image obtained from the encoded data) into the variable m (step S2). M set in step S2 is used for the control in steps S3 to S7.

  After step S2, the first determination unit 14 determines whether m <N is satisfied (step S3). If the variable m is less than N (Y in Step S3), the first determination unit 14 determines the LL component (mLL component) with the decomposition level number m and the xx component (1xx to mxx component) with the decomposition level number 1 to m. The code amount when the intermediate code is encoded is estimated. At this time, the first determination unit 14 generates a Huffman table (see FIG. 17) from the frequency tables of Y, U, and V of these components, and for each Huffman code, the bit length of the Huffman code, Multiply the corresponding occurrence frequency of Y, U, or V, and calculate the sum of the multiplication results. Furthermore, in this example, since len is included in the intermediate code, the code amount when each len included in the intermediate code is converted into a code (for example, a gamma code) is also added to the sum. The 1st determination part 14 calculates | requires this sum for every component, and calculates those sum total (step S4). This total value is an estimated code amount when the intermediate code of the mLL component and the 1xx to mxx components is encoded. The estimated code amount is a value estimated as the data amount after encoding the intermediate code.

  Subsequently, the first determination unit 14 determines whether or not the estimated code amount is larger than the target code amount of the region to be encoded (step S5). Thus, the target code amount calculated by the region-specific target code amount calculation unit 4 is used as a threshold value in step S5. It is also used as a threshold value in steps S13 and S19 described later.

  When the estimated code amount is less than or equal to the target code amount (N in step S5), the process proceeds to step S9. In step S9, the encoding unit 17 converts the MLL intermediate code into a Huffman code. The encoding unit 17 may convert the Y, U, V of the mLL intermediate code into the corresponding Huffman code using the Huffman table corresponding to Y, U, V of the mLL component generated in step S4 (step S4). S9). Further, the encoding unit 17 converts the intermediate codes of the components mxx, (m−1) xx,..., 1xx into Huffman codes (step S10). The intermediate codes Y, U, and V from mxx to 1xx may be converted into corresponding Huffman codes using the Huffman table generated in step S4. In addition, in steps S9 and S10, the encoding unit 17 encodes len in each intermediate code into, for example, a gamma code.

  When the estimated code amount is larger than the target code amount (Y in step S5), the first determination unit 14 increments the value of the variable m by 1 (step S7), and repeats the processes after step S3. When the variable m is incremented by 1 and the process proceeds from step S3 to step S4 again, it is not necessary to obtain the code amount again for the component for which the code amount was obtained in the previous step S4, and the mLL component indicated by m after the increment And the mxx component are obtained, and the total code amount of the mLL component and the 1xx to mxx components may be estimated.

  If it is determined in step S3 that m = N (N in step S3), requantization processing (step S8) is performed. The process after step S11 shown in FIG. 20 represents the requantization process of step S8. That is, if m = N in step S3, the process proceeds to step S11 (see FIG. 20).

  The transition to step S11 means that the code estimation amount when the LL component of the intermediate code of the decomposition level number “N−1” and the xx component of the intermediate code from the decomposition level number 1 to “N−1” is encoded. Is larger than the target code amount. At this time, the first determination unit 14 causes the encoding unit 17 to encode the NLL component (step S11). The encoding unit 17 generates a Huffman table from the frequency tables of Y, U, and V of the NLL intermediate code, and converts the Y, U, and V into corresponding Huffman codes. The encoding unit 17 encodes each len included in the NLL intermediate code into, for example, a gamma code.

  Next, the 1st determination part 14 substitutes N to the variable m (step S12). M set in step S12 is used for the control in steps S13 to S21.

After step S12, the first determination unit 14 determines whether m ≦ m _q is _satisfied (step S13). If m ≦ m _q does not hold (N in step S13), the code amount of the xx component (that is, the mxx component) of the number of decomposition levels indicated by the variable m is estimated (step S14). In step S14, a Huffman table is generated from the frequency tables of Y, U, and V of the mxx component, and for each Huffman code, the bit length of the Huffman code is multiplied by the occurrence frequency of the corresponding Y, U, or V, The sum of the multiplication results is calculated. The bit length when each len is encoded may be added to the sum. If the estimated code amount of the mxx component has been calculated in step S4, the estimated code amount may be referred to.

  Subsequently, the first determination unit 14 determines whether or not the sum of the code estimation amount obtained in step S14 and the data amount of already encoded code data is larger than the target code amount ( Step S15). If the sum is larger than the target code amount (Y in step S15), the process is terminated. If the sum is equal to or less than the target code amount (N in step S15), the encoding unit 17 encodes the mxx component (step S16). The encoding unit 17 converts the mxx component Y, U, and V into the corresponding Huffman code using the Huffman table corresponding to the mxx component Y, U, and V used for the code amount estimation in step S14. The encoding unit 17 encodes len in the mxx component into, for example, a gamma code.

Subsequently, the first determination unit 14 subtracts 1 from the value of the variable m (step S17), and repeats the processing after step S13. By repeating the loop processing from step S13 to step S17, the first determination unit 14 selects the number of decomposition levels in a predetermined range (that is, m _q +1 to N) in descending order every time the process proceeds to step S14. Then, the code amount of the xx component is estimated.

If it is determined in step S17 that the value of the variable m is subtracted by 1 and m ≦ _mq is determined in step S13 (Y in step S13), the process proceeds to step S18. Therefore, after step S18, the value of m is _mq .

  In step S18, the second determination unit 8 encodes the intermediate code of each xx component from the xx component (that is, the mxx component) of the decomposition level number indicated by m at the time of transition to step S18 to the 1xx component. Estimate the amount. When the process first proceeds to step S18 (that is, when the requantization of step S20 has not been performed yet), the code amount when the intermediate code of each component of mxx to 1xx is encoded is estimated. When the process proceeds to step S18 after the requantization process in step S20, the code amount of the intermediate code of each component of mxx to 1xx after the requantization is estimated.

  When estimating the code amount of the intermediate code of each component, a Huffman table is generated from the frequency tables of Y, U, and V of each component, and for each Huffman code, the bit length of the Huffman code and the corresponding Y , U or V occurrence frequency, the sum of the multiplication results is calculated, and the bit length when each len is encoded may be added to the sum.

  In the requantization in step S20, a frequency table of Y, U, and V after requantization is also created. When the process proceeds from step S20 to step S18, a Huffman table may be generated using the frequency table.

  After step S18, the second determination unit 8 determines that the sum of the code amount obtained by encoding mxx to 1xx obtained in step S18 and the data amount of the already encoded code data is larger than the target code amount. It is determined whether or not it is larger (step S19).

  If the sum is larger than the target code amount, the second determination unit 8 determines to perform re-quantization, and proceeds to step S20 (Y in step S19).

  If the total is equal to or less than the target code amount, the second determination unit 8 determines not to re-quantize, and proceeds to step S21 (N in step S19).

  In step S21, the encoding unit 17 converts the Yxx, Uxx, and 1xx components from the mxx component into the corresponding Huffman codes using the Huffman table used for the code amount estimation in step S18. The encoding unit 17 encodes len in each component into, for example, a gamma code.

In step S20, the requantization unit 16 requantizes Y, U, and V from the 1xx intermediate code to the mxx intermediate code. Since m in step S18 and thereafter is m _q , Y, U, and V from the 1xx intermediate code to the m _q xx intermediate code are requantized. When performing the requantization process, the requantization unit 16 newly determines, for example, a Q step that is already selected Q step or higher, or a dead zone that is already selected dead zone or higher.

In step S20, the requantization unit 16 determines a new dead zone for each component from 1xx to m _q xx. At this time, the re-quantization unit 16 sets dz + h · qs as a new dead, assuming that h is an integer greater than or equal to 0, and the dead zone and Q step used when the intermediate code is generated in step S1 are dz and qs, respectively. Set in the zone. Then, the requantization unit 16 requantizes the value of the intermediate code (Y, U, V) whose absolute value is equal to or less than the new dead zone to 0.

Further, the requantization unit 16 determines a new Q step for each component from 1xx to m _q xx. At this time, it is preferable that the requantization unit 16 sets qs · (2n + 1) as a new Q step using the above qs, where n is an integer of 0 or more. Then, the range in which the absolute value exceeds the new dead zone is divided for each new Q step (qs · (2n + 1)) to determine the range. Then, the intermediate codes (Y, U, V) of each component from 1xx to m _q xx belonging to each range are quantized to the median value of the range. The reason why it is preferable to set the new Q step to qs · (2n + 1) is that the Q value is determined in this way, so that the median value of the newly determined range is the center of the range used when quantizing in step S1. This is because the image quality of the encoded data can be prevented from deteriorating because it matches the value. The new range in which the Q step is defined as qs · (2n + 1) is a range obtained by combining the ranges used when the step S1 is quantized.

  However, if the Q step for deriving the intermediate code in step S1 is 1, an arbitrary value (however, an integer of 1 or more) may be set as a new Q step in step S20. If the Q step in step S1 is 1, the median value of the newly defined range matches the median value of the range used when quantizing in step S1, regardless of the value of the newly defined Q step. It is.

  In step S20, the second determination unit 15 generates new Y, U, and V frequency tables obtained by requantizing the intermediate code. The ranges used for requantization are individual ranges determined by dividing an absolute value of the dead zone or less by a new Q step. Each of these ranges is a combination of the ranges used when generating the intermediate code in step S1, and corresponds to the range used for quantization in step S1. Then, the frequency table in each range (for example, a range to be quantized to 0, a range to be quantized to 1, etc.) in step S1 has already been derived in step S1. Therefore, the second determination unit 15 specifies each range used in step S1 corresponding to each range defined for quantization in step S20, and newly calculates the sum of the occurrence frequencies in each range. The occurrence frequency may be within a certain range. By determining the occurrence frequency in this way, it is not necessary to recount the occurrence frequencies of Y, U, and V quantized in a new range.

  FIG. 21 is an explanatory diagram schematically showing frequency table derivation in step S20. The upper histogram shown in FIG. 21 indicates the frequency of occurrence of data quantized to the median value of each range (for example, Y) in the quantization for generating the intermediate code in step S1. Further, the lower histogram shown in FIG. 21 indicates the frequency of occurrence of data quantized to the median value of each range in the requantization in step S20. In FIG. 21, A to I and S to U are examples of median values after quantization. In the requantization process in step S20, the requantization unit 16 divides the range from the new Q step and the dead zone, and as a result, combines three ranges with A, B, and C as median values, It is assumed that one range as the median value is defined and data (Y) belonging to the range is quantized to the median value S. In this case, the second determination unit 15 uses the number of data quantized to value A, the number of data quantized to value B, and the quantum to value C in the frequency table represented as the upper histogram in FIG. The sum of the number of converted data may be calculated, and the frequency of occurrence of data that belongs to a new range (a range with S as the median) corresponding to the three ranges and is quantized to the value S may be calculated. The occurrence frequency in other ranges may be calculated similarly. By calculating the occurrence frequency after requantization in this way, it is not necessary to recount the occurrence frequency of data in each range, so that the processing can be speeded up.

  The same applies to the occurrence frequency of data in other ranges. As described above, if the new dead zone is dz + h · qs and the new Q step is qs · (2n + 1), the median in the new range is the median in the quantization at the time of intermediate code generation. Will be equal. For example, in the example shown in FIG. 21, S = B, T = E, and U = H.

  After the above step S20, the processes after step S18 are repeated. The second determination unit 15 newly generates a frequency table in step S20. In the subsequent step S18, the second determination unit 15 uses the frequency table to generate a Huffman table indicating the sign of the value quantized in step S20. At this time, the second determination unit 15 uses the new value in the Huffman table generated from the new frequency table for each value after quantization in Step S1 (that is, each intermediate code generated in Step S1). Should be associated with.

  FIG. 22 is an explanatory diagram showing the association between the value of the intermediate code and the code derived after requantization. For example, it is assumed that the intermediate code −1, 0, 1 before re-quantization illustrated in FIG. 22 is re-quantized to 0. In this case, the second determination unit 15 derives a Huffman table from the frequency table of each value such as 0 after requantization. In this Huffman table, it is assumed that the code corresponding to the intermediate code 0 is 0. Then, the second determination unit 15 associates a new code 0 with each of the original intermediate codes −1, 0, 1. For example, each code “100”, “0”, “101” corresponding to the original intermediate code-1, 0, 1 is replaced with a new code “0”. By processing in this way, the same code as 0 can be referred to from the original intermediate codes -1,1. Then, a Huffman code corresponding to the value after requantization can be generated without rewriting the intermediate code.

  By performing the requantization process, the number of bits of the Huffman code can be reduced. Therefore, the total code amount after encoding can be reduced by repeating the loop processing of steps S18 to S20. Then, when the estimated total code amount becomes equal to or less than the target code amount, the process proceeds to step S21 and the encoding process may be performed.

  In addition, as a result of increasing the dead zone value in the requantization process in step S20, all intermediate codes (Y, U, V) may be quantized to one value (0). In this case, the encoding unit 17 may prohibit the encoding process in step S21. That is, when all the intermediate codes are quantized to 0 by requantization, the encoding unit 17 ends the process without converting the values of Y, U, and V after the requantization into Huffman codes. It's okay. Also, the process of encoding len need not be executed. As described above, when all of Y, U, and V are quantized to 0, the processing amount can be reduced by not performing the encoding process.

In the encoding method as described above, when the first determination unit 14 determines to encode the xx component for the number of decomposition levels (m _q +1 to N) in a predetermined range, the xx component is Encode it. Then, the intermediate code of the decomposition level number (1 to m _q ) is requantized according to the code amount when the decomposition level number (1 to m _q ) lower than the decomposition level number is encoded. Therefore, since re-quantization is performed for a part of the number of decomposition levels obtained by the wavelet transform, it is possible to control the code amount to be equal to or less than the target code amount while suppressing the processing load.

  In the requantization process, a range obtained by combining the dead zone used when deriving the intermediate code and the range determined from the Q step is determined, and the value in each new range is quantized to the median value of the range. In this way, the new range is obtained as a range combining the range at the time of deriving the intermediate code, and therefore, when obtaining a new frequency table, the sum of the results counted at the time of deriving the intermediate code may be obtained (see FIG. 21). . Therefore, the processing load can be suppressed and the processing speed can be increased.

  In addition, by defining the Q step in the requantization process as qs · (2n + 1), the value after requantization can be made to match the value of the intermediate code, and deterioration of the image quality of the encoded data can be prevented. it can.

  As already described, if the data amount of the image data before encoding in the region where the image is transmitted is equal to or less than a predetermined threshold, the region-specific target code amount calculation unit 4 The maximum value that the target code amount can take may be set. In this case, it is always determined in step S5 that the estimated code amount is equal to or less than the target code amount, and the process proceeds to step S9. As a result, in a small area where the data amount is equal to or less than a predetermined threshold, the compression rate is lowered and the image quality is improved.

  In the above description, when intermediate data is generated (for example, in step S1), the result of quantizing the coefficient obtained by the wavelet transform is described as an example of an intermediate code. Coefficients (wavelet coefficients) obtained by the conversion may be used as intermediate codes. Specifically, the wavelet coefficient itself may be used as intermediate data for a component having a predetermined number of decomposition levels or more. For example, when the wavelet transform is performed up to the decomposition level number N, the wavelet coefficients having the decomposition level number “N−T” or more and N or less are used as intermediate codes, and the wavelet coefficients less than the decomposition level number “N−T” are processed in step S1. As described above, the result of quantizing the wavelet coefficients may be used as the intermediate code. The number of decomposition levels “N−T” to “T” using wavelet coefficients as intermediate codes as they are may be determined in advance according to the image quality and compression rate required for the encoded data. The wavelet coefficient may be used as an intermediate code for the LL component, and the data obtained by quantizing the wavelet coefficient as described in step S1 may be used as the intermediate code for the xx component.

  In the embodiment described above, when there is an encoding completion area in which encoding of image data is completed among the areas where image data is transmitted, the total amount of image data after encoding in the encoding completion area is calculated. There are subtracting means for subtracting from the overall target code amount, and the compression rate calculating means performs subtraction by the subtracting means, and there are areas where image data is not yet encoded among the areas where image data is transmitted. In this case, the overall target code amount after subtraction by the subtracting unit is divided by the total amount of image data before encoding in a region where encoding is not performed, thereby calculating a compression rate, and target code amount calculating unit for each region Is obtained by multiplying the amount of image data before encoding in an unencoded area by the compression rate calculated by the compression rate calculating means to encode the image data in the unencoded area. Eyes when turning Configured to calculate the amount of codes is disclosed.

  In the above embodiment, the target code amount calculation unit for each region is a region where the image data is not encoded, and the data amount of the image data before encoding is a predetermined threshold value or less. On the other hand, a configuration is disclosed in which the target code amount is defined as the maximum value that the target code amount can take.

  Further, the above embodiment includes a region encoding unit that encodes image data to a code amount equal to or less than the target code amount determined by the region-specific target code amount calculation unit for each region to which the image data is transmitted. A configuration is disclosed.

  Further, the above embodiment discloses a configuration including an area specifying unit that specifies a rectangular area including an updated area in one frame as an area to which image data is transmitted.

  The present invention is suitably applied to, for example, a target code amount calculation apparatus that calculates a target code amount used in an encoding process for encoding image data so that the encoded data amount is equal to or less than the target code amount. The

It is a schematic diagram which shows the example of the area | region where image data is encoded and transmitted. It is a block diagram which shows 1st Embodiment of the target code amount calculation apparatus of this invention. It is a flowchart which shows an example of the process progress of the target code amount calculation apparatus of this invention. It is a schematic diagram which shows the example of the area | region where image data is encoded and transmitted. It is a block diagram which shows 2nd Embodiment of the target code amount calculation apparatus of this invention. It is a flowchart which shows the example of a target code amount recalculation process. It is a block diagram which shows the structural example in the case of providing an area | region specific part. It is explanatory drawing which shows the example of the area | region where an image is transmitted within 1 frame. It is explanatory drawing which shows the example of the head and the end of an update line. It is explanatory drawing which shows the example of the left end of an update column, and a right end. It is explanatory drawing which shows the example of the range enclosed by the head and tail of an update line, and the left end and right end of an update column. It is a block diagram which shows the structural example in the case of providing an area | region encoding part. It is a block diagram which shows the example of an area | region encoding part. It is explanatory drawing which shows a wavelet transformation typically. It is explanatory drawing which shows typically the intermediate code produced | generated from the wavelet coefficient. It is explanatory drawing which shows the example of a frequency table. It is explanatory drawing which shows the example of a Huffman table. It is explanatory drawing which shows the example of the result of a Huffman encoding typically. It is a flowchart which shows an example of the process progress of an area | region encoding part. It is a flowchart which shows an example of the process progress of an area | region encoding part. It is explanatory drawing which shows frequency table derivation typically. It is explanatory drawing which shows matching with the code | cord | chord derived after the value of the intermediate code and requantization.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Target code amount calculation apparatus 2 Overall target code amount calculation part 3 Compression rate calculation part 4 Target code amount calculation part according to area | region 5 Subtraction part 6 Area | region specific | specification part 7 Area | region encoding part

Claims (9)

A target code amount calculation device for calculating a target code amount when encoding image data in an area within one frame and where image data is transmitted via a communication network,
An overall target code amount calculating means for calculating an overall target code amount, which is a target code amount for the entire region in which image data is transmitted, by dividing the communication band of the communication network by a predetermined frame rate;
Compression rate calculation means for calculating a compression rate when encoding image data by dividing the overall target code amount by the total data amount of image data before encoding of each region in which the image data is transmitted When,
For each area where image data is transmitted, the image data of the area is encoded by multiplying the data amount of the image data before encoding of the area by the compression ratio calculated by the compression ratio calculation means. And a target code amount calculation unit for each region that calculates a target code amount at that time. When there is an encoding completion area in which encoding of image data has been completed in each area to which image data is transmitted, the total amount of image data after encoding in the encoding completion area is subtracted from the overall target code quantity. With subtraction means,
The compression rate calculation means performs subtraction by the subtraction means, and when there is an area in which image data is not yet encoded among the areas to which image data is transmitted, By dividing the overall target code amount by the total amount of image data before encoding in a region that has not been encoded, the compression rate is calculated,
The target code amount calculation unit for each region performs encoding by multiplying the data amount of image data before encoding in an unencoded region by the compression rate calculated by the compression rate calculation unit. The target code amount calculation device according to claim 1, wherein a target code amount is calculated when encoding image data of an undisclosed area. The region-specific target code amount calculating means includes:
For a region in which image data is not encoded and the amount of image data before encoding is equal to or less than a predetermined threshold, the target code amount is set to the maximum value that the target code amount can take. The target code amount calculation apparatus according to claim 1 or 2. The region coding means for coding the image data to a code amount equal to or less than the target code amount determined by the region-specific target code amount calculation means for each region in which the image data is transmitted. The target code amount calculation apparatus according to any one of the above. The target code amount calculation according to any one of claims 1 to 4, further comprising region specifying means for specifying a rectangular region including an updated region in one frame as a region to which image data is transmitted. apparatus. A target code amount calculation method for calculating a target code amount when encoding image data in an area within one frame and where image data is transmitted via a communication network,
By dividing the communication bandwidth of the communication network by a predetermined frame rate, a total target code amount that is a target code amount for the entire region where image data is transmitted is calculated,
By dividing the overall target code amount by the total data amount of the image data before encoding of each region in which the image data is transmitted, a compression rate when encoding the image data is calculated,
For each region where image data is transmitted, the target code amount when encoding the image data of the region is calculated by multiplying the data amount of the image data before encoding of the region by the calculated compression rate. A target code amount calculation method, characterized by: calculating. When there is an encoding completion area in which encoding of image data has been completed in each area to which image data is transmitted, the total amount of image data after encoding in the encoding completion area is subtracted from the overall target code quantity. ,
After the subtraction, if there is a region where the image data is not yet encoded among the regions where the image data is transmitted, the code of the region where the overall target code amount after the subtraction is not encoded Calculate the compression rate by dividing by the total amount of image data before conversion,
The target code amount when encoding the image data in the non-encoded area by multiplying the data amount of the image data before encoding in the non-encoded area by the calculated compression rate The target code amount calculation method according to claim 6. A target code amount calculation program installed in a computer for calculating a target code amount when encoding image data in an area within one frame and where image data is transmitted via a communication network,
In the computer,
An overall target code amount calculation process for calculating an overall target code amount, which is a target code amount for the entire region in which image data is transmitted, by dividing the communication band of the communication network by a predetermined frame rate;
A compression rate calculation process for calculating a compression rate when encoding image data by dividing the overall target code amount by the total data amount of image data before encoding of each region in which the image data is transmitted ,and,
For each area where image data is transmitted, the image data of the area is encoded by multiplying the data amount of the image data before encoding of the area by the compression ratio calculated in the compression ratio calculation process. A target code amount calculation program for executing a region-specific target code amount calculation process for calculating a target code amount at the time. In the computer,
When there is an encoding completion area in which encoding of image data has been completed in each area to which image data is transmitted, the total amount of image data after encoding in the encoding completion area is subtracted from the overall target code quantity. Subtraction processing,
After the subtraction process, when there is an area where the image data is not yet encoded among the areas where the image data is transmitted, the overall target code amount after the subtraction process is not encoded. By dividing by the total amount of image data before encoding the region, the compression rate is recalculated to calculate the compression rate, and the data amount of the image data before encoding in the region that has not been encoded is A region-specific target code amount recalculation process for calculating a target code amount for encoding image data in a region that has not been encoded by multiplying the compression rate calculated in the rate recalculation processing. Item 9. The target code amount calculation program according to Item 8.