JP2002176560A - Picture processor, picture processing method, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily control the encoding volume after hierarchical encoding processing without complicated processing by one encoding processing. SOLUTION: An approximate curve indicating the relation between a hierarchical level and a code volume is obtained by a discrimination part in accordance with code volumes after encoding processing in hierarchical levels A and B calculated by a variable length code volume calculation part in a code volume monitor part 209. A hierarchical level according with a target code volume is forecast and determined on the basis of this approximate curve, and a color picture encoding part 104 outputs encoded data obtained by encoding processing on the basis of this hierarchical level, and thus the hierarchical level in which the code volume after hierarchical encoding processing is the target code volume is forecast and determined by one encoding processing without performing encoding processing plural times to control the code volume after hierarchical encoding processing in the color picture encoding part 104.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and a recording medium, and more particularly to an image processing apparatus suitable for use in an image processing apparatus capable of switching encoding processing for compressing an image for each image area. Things.

[0002]

2. Description of the Related Art Conventionally, as a coding method for compressing a color still image, a JPEG method using discrete cosine transform and a method using Wavelet transform have been often used. These encoding systems are variable-length encoding systems in which the amount of code after encoding is basically different according to the original image to be encoded.

[0003]

However, in the international standardized JPEG system, which is one of these systems, only one set of quantization matrices can be defined for an image to be coded. Therefore, in order to control the code amount after encoding to be constant, an encoding process is repeatedly performed by prescan or the like, and an appropriate set of quantization matrices is determined by predicting the code amount after encoding. Was. Therefore, there is a problem that a plurality of encoding processes must be performed in order to control the code amount after encoding to be constant.

[0004] In an image, there are an area where deterioration of image quality due to encoding is relatively conspicuous and an area where the deterioration is not so noticeable. For example, in an image composed of a character area and a photographic image area other than characters, the image quality of the character area is easily noticeable, while the image quality of the photographic image area is not so noticeable. Further, for example, in an image composed of a computer image area and a natural image area, the computer image area tends to have a noticeable deterioration in image quality, but the natural image area has a low image quality deterioration. Therefore, by reducing the quantization step in an area where image degradation is conspicuous and coarsening the quantization step in an area where image degradation is not conspicuous, the code amount after encoding can be reduced and the image quality can be improved.

However, in the above-described JPEG system using the discrete cosine transform and the system using the Wavelet transform, the entire image is uniformly encoded. For this reason, as described above, it has not been possible to improve the image quality by reducing the amount of code after encoding by dividing the region into regions where the image quality deterioration is relatively conspicuous and regions where the image quality deterioration is not so noticeable.

[0006] Also, many of the conventional region division codings are region division coding using an irreversible coding method.
Few have combined this with a lossless coding scheme. Here, the reversible coding method is a method capable of faithfully reproducing the original image even if the original image is once encoded and then decoded. It is known that when trying to increase the coding efficiency in each of the lossless coding method and the irreversible coding method, the commonality between the lossless coding method and the irreversible coding method decreases. Specifically, in the lossless coding method, coding efficiency can be increased by using predictive coding that predicts a target pixel from surrounding pixels, and in the lossy coding method, discrete coding cosine transform is used. The encoding efficiency can be increased by combining the optimal transform encoding and the entropy encoding of the transform coefficients.

[0007] However, when switching between the lossless coding method and the irreversible coding method while increasing the coding efficiency, two different coding methods such as predictive coding and discrete cosine transform coding are used. It was necessary to use.

As a method for solving this problem, a coding efficiency is slightly lowered but a reversible discrete cosine transform method (Kunori Komatsu et al., 'Reversible Discrete Cosine Transform and Its Application to Image Information Compression', IEICE Technical Report IE9
7-83, 1997.11). In this method, the N-point reversible transform is performed by a discrete cosine transform (DC
A method applied to T: Discrete Cosine Transform has been proposed, and the irreversible transform is realized by performing bit plane coding of DCT coefficients. However, it has not been put to practical use because the above-mentioned operation in the reversible transform is performed by many floating-point operations and there is an operation error with a coefficient in DCT operation using a normal real number. Was. Further, since the method is for one image, the use of this method is also limited.

When switching between the lossless encoding method and the irreversible encoding method, in order to convert from lossless encoding to irreversible encoding, encoded data encoded by lossless encoding is used. There is a problem that a complicated process of decoding and returning to the original image data and performing an encoding process by irreversible encoding is required.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and has been made to perform a plurality of encoding processes or to decode encoded encoded data to further encode the encoded data. An object of the present invention is to make it possible to easily control the amount of coding after hierarchical coding processing in one coding processing without performing complicated processing.

[0011]

An image processing apparatus according to the present invention is an image processing apparatus having a function of performing hierarchical coding processing of image data by bit-plane coding. Code amount calculating means for calculating the code amount in each of a plurality of layers; and code amount predicting means for predicting the code amount after the hierarchical coding process according to the number of layers based on the code amount calculated by the code amount calculating means. And encoding means for performing hierarchical encoding processing on the image data according to a prediction result by the code amount predicting means.

According to another feature of the image processing apparatus of the present invention, the code amount predicting means determines the number of layers and the code after hierarchical coding processing based on the code amount calculated by the code amount calculating means. First to find an approximate curve showing the relationship with quantity
, And predicts the code amount after the hierarchical coding process according to the number of layers based on the approximate curve obtained by the first approximate curve calculating means.

Another feature of the image processing apparatus according to the present invention is that the code amount predicting means is configured to decode each of the image data and the image data having undergone the hierarchical coding process in a plurality of hierarchies. Error calculating means for calculating a least square error with the image, based on the least square error calculated by the error calculating means, the code amount after hierarchical coding processing of the image data to obtain a target image quality It is characterized by predicting.

According to another feature of the image processing apparatus of the present invention, the code amount predicting means calculates a difference between the number of layers and the least square error based on the least square error calculated by the error calculating means. A second approximation curve calculating means for obtaining an approximate curve indicating a relationship, and a hierarchical encoding process of the image data for obtaining a target image quality based on the approximate curve calculated by the second approximate curve calculation means. It is characterized by predicting the amount of code later.

Another feature of the image processing apparatus of the present invention is that the encoding means divides the image data into a plurality of tile images and performs a hierarchical encoding process on each of the divided tile images. Another feature of the image processing apparatus according to the present invention is that the encoding unit divides the image data into regions and performs a hierarchical encoding process according to an image region flag indicating an attribute of the region-divided image data. It is characterized by.

According to another feature of the image processing apparatus of the present invention, the image area flag is a flag for identifying a character area and a photographic image area, and the encoding means determines that the image area flag is a character area. It is characterized in that lossless encoding processing is performed when the flag is an area, and irreversible encoding processing is performed when the image area flag is a flag of a photographic image area.

Another feature of the image processing apparatus of the present invention is that, when the prediction result by the code amount predicting means is larger than a predetermined code amount, the coding means performs the lossless encoding processing by the irreversible coding processing. It is characterized by switching to encoding processing.

Another feature of the image processing apparatus of the present invention is an image processing apparatus having a function of performing hierarchical coding processing on image data by bit plane coding, wherein the image data is converted into a character area and a photographic image. In accordance with an image area flag for identifying the character area and the photographic image area, lossless encoding processing is performed in the character area, and irreversible encoding processing is performed in the photographic image area. Features.

An image processing method according to the present invention is an image processing method for hierarchically encoding image data by bit-plane encoding, wherein the amount of code of the image data after the hierarchical encoding is calculated for each of a plurality of layers. Then, based on the calculated code amount after the hierarchical coding process, predicting the code amount after the hierarchical coding process according to the number of layers, and performing the hierarchical coding process on the image data according to the result. Features.

Another feature of the image processing method of the present invention is that an approximate curve indicating the relationship between the number of layers and the code amount after the above-mentioned layer coding processing is obtained, and the hierarchical coding according to the number of layers is performed. It is characterized in that the code amount after processing is predicted.

Another feature of the image processing method of the present invention is that the least square error between the image related to the above image data and each decoded image related to the image data after the hierarchical coding processing in a plurality of layers is calculated. Based on the calculated least square error, a code amount of the image data after the hierarchical coding process for obtaining a target image quality is predicted.

Another feature of the image processing method of the present invention is that an approximate curve indicating the relationship between the number of layers and the least square error is obtained, and the hierarchical code of the image data for obtaining a target image quality. It is characterized by predicting the code amount after the conversion processing.

Another feature of the image processing method of the present invention is that the image data is divided into a plurality of tile images, and each of the image data is hierarchically encoded. Another feature of the image processing method of the present invention is that the image data is divided into regions, and hierarchical coding is performed according to an image region flag indicating an attribute of the region-divided image data.

Another feature of the image processing method of the present invention is that the image area flag is a flag for identifying a character area and a photographic image area, and is reversible when the image area flag is a character area flag. An encoding process is performed, and when the image area flag is a flag of a photographic image area, an irreversible encoding process is performed.

Another feature of the image processing method of the present invention is that, when the code amount after the hierarchical coding process is predicted to be larger than a predetermined code amount, the lossless coding process is performed by the irreversible code. It is characterized by switching to a conversion process.

Another feature of the image processing method of the present invention is an image processing method for hierarchically encoding image data by bit plane encoding, wherein the image data is divided into a character area and a photographic image area. According to an image area flag that divides the area and identifies the character area and the photographic image area, lossless encoding processing is performed in the case of the character area, and irreversible encoding processing is performed in the case of the photographic image area. .

The computer-readable recording medium of the present invention is characterized in that a program for causing a computer to function as each of the above means is recorded. Another feature of the computer-readable recording medium of the present invention is that a program for causing a computer to execute the procedure of the image processing method is recorded.

According to the present invention configured as described above, the code amount after hierarchical coding of image data is calculated for each of a plurality of layers, and the calculated code amount after hierarchical coding is calculated based on the calculated code amount after hierarchical coding. Since the code amount after the hierarchical coding process according to the number is predicted and the hierarchical coding process is performed according to the result, the coding amount after the coding process can be easily controlled by one coding process. Become like

Also, an approximate curve indicating the relationship between the number of layers and the amount of code after the hierarchical coding process is obtained from the calculated code amount after the hierarchical coding process, and the approximate curve indicating the relationship between the number of layers and the hierarchical coding process When the code amount is predicted, the number of layers for encoding the image data can be controlled according to the prediction result. Further, a least square error between the image related to the image data and each decoded image related to the image data after the hierarchical coding process in a plurality of layers is calculated, and the number of layers is calculated based on the calculated least square error. When an approximate curve indicating the relationship with the least square error is obtained and the code amount of the image data after the hierarchical coding process for obtaining the target image quality is to be predicted, the image related to the image data and the hierarchical It becomes possible to encode the image data with a code amount such that the least square error between the encoded image data and the decoded image is smaller than a predetermined value.

When the image data is divided into regions and the encoding process is performed in accordance with an image area flag indicating the attribute of the image data, a region where image quality deterioration is conspicuous and a region where image quality deterioration is not conspicuous are distinguished from each other. Encoding processing can be performed so as to have an appropriate code amount. Also,
When the code amount after encoding is predicted to be larger than the predetermined code amount, when the lossless encoding process is switched to the irreversible encoding process, the code amount can be reduced, and Similarly, when the number of hierarchies for encoding image data is controlled, the code amount can be reduced.

[0031]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a block diagram showing a configuration example of a digital image device to which an image processing device according to a first embodiment is applied.

In FIG. 1, reference numeral 101 denotes an image input unit;
2 is a block buffer for compression, 103 is an image area flag encoding unit, 104 is a color image encoding unit, 105 is a compression memory, 106 is a large-capacity external storage device, 107 is a decoding memory, and 108 is an image area flag decoding unit , 109 are a color image decoding unit, and 110 is a development block buffer. Also one
Reference numeral 11 denotes an interface (not shown),
For example, it is connected to a printer or the like.

Reference numeral 112 denotes an image scanner for supplying image data S1 and an image area flag S2 of an image to be encoded. Reference numeral 113 denotes a page description language rendering unit (hereinafter, referred to as a "PDL rendering unit") for supplying image data S3 and an image area flag S4 of an image to be encoded.

The image input unit 101 receives the image signals S1, S3 and the image area flags S2, S supplied from the image scanner unit 112 and the PDL rendering unit 113.
4 is selectively supplied by the selector 114 in accordance with the switching signal S5. For example, in the case of a digital copying machine, the image data S1 and the image area flag S2 supplied from the image scanner unit 112 are selected by the selector 114, and in the case of a printer, the PDL rendering unit 1 is selected.
13 and the image area flag S
4 is selected by the selector 114 and supplied to the image input unit 101.

The image input unit 101 supplies the image data and the image area flag selected and supplied by the selector 114 to the compression block buffer 102. The compression block buffer 102 divides the supplied image data and the image area flag into tile images each having a predetermined size of M × M pixels. Here, the image area flag is attached to each pixel. In the present embodiment, as described later, encoding using discrete cosine transform (hereinafter referred to as “DCT”) is performed for each block of 8 × 8 pixels. Since the processing is performed, an image area flag that is representative for each block of 8 × 8 pixels is used.

The compression block buffer 102
The tile image divided for each × M pixels is supplied to the color image encoding unit 104 and the image area flag encoding unit 103. The color image encoding unit 104 performs DCT encoding (JPEG), which is encoding of color image information, and the image region flag encoding unit 103 performs run-length encoding, which is encoding of image region flag information. Is done.

At this time, the image area flag encoding unit 103
In accordance with the supplied image area flag and an instruction from the monitoring unit 115, which will be described later, an instruction is made to the color image encoding unit 104 for encoding parameters for encoding the color image information.
The color image encoding unit 104 switches the encoding parameters according to the instruction and performs an encoding process. In addition,
In the present embodiment, the encoding parameter is the number of levels of hierarchical encoding. Encoded data encoded by the image area flag encoding unit 103 and the color image encoding unit 104 is transmitted to the external storage device 106 via the compression memory 105.
Is stored.

When decoding the encoded data stored in the external storage device 106, the image area flag decoding unit 108 decodes the image area flag information of the encoded data supplied via the decoding memory 107. I do. In accordance with the decoding result of the image area flag decoded by the image area flag decoding unit 108, the color image decoding unit 109 switches the decoding parameter to switch the color parameter of the encoded data supplied through the decoding memory 107. Decrypt the information. Then, the encoded data is decoded by supplying the decoded image data to the expansion line buffer 110.
In the present embodiment, the decoding parameter is the number of levels of hierarchical coding.

Reference numeral 115 denotes a monitor, which monitors the code amount of the coded data which is coded by the image area flag coder 103 and the color image coder 104 and supplied to the compression memory 105. Then, when it is predicted that the code amount exceeds the preset code amount, the monitoring unit 125 changes the image area flag to the high compression area and changes the image area flag to the image area flag encoding unit 10.
3, the image area flag encoding unit 103 instructs the color image encoding unit 104 to perform high compression encoding.
In this way, the color image coding unit 104 controls the layer level of the layer coding.

FIG. 2 is a diagram for explaining the processing of the color image encoding unit 104 and the color image decoding unit 109 shown in FIG. In FIG. 2, S21 is an image signal supplied from an image input unit (not shown), and in the case of a color signal, a signal of three colors of red (R), green (G), and blue (B), each having 256 gradations It is.

Reference numeral 104 denotes the color image encoding unit shown in FIG.
It is referred to as “DCT unit”. ) 202, a bit plane layer coding unit 203 and a selector 204.

The color converter 201 converts the supplied image signal (RGB signal) S21 into a luminance / color difference signal (YCbCr signal). The DCT unit 202 is a part of the color converter 20
1 is subjected to spatial frequency transform (DCT transform) for each of the luminance and chrominance signals supplied. The bit plane layer coding unit 203 performs DCT by the DCT unit 202.
The converted signal is subjected to a bit-plane hierarchical encoding process and supplied to the selector 204. The selector 204 is controlled by the tile attribute determining unit 210 and controls the amount of hierarchically encoded data stored in the memory 215.

The memory 205 stores the hierarchically encoded data that has been encoded by the bit plane hierarchical encoding unit 203. The memory 205 includes the compression memory 105, the external storage device 106, and the decoding memory 107 shown in FIG.
Is equivalent to

Reference numeral 109 denotes the color image decoding unit shown in FIG. 1, which includes a bit plane hierarchy decoding unit 206, an inverse discrete cosine transform unit (hereinafter, referred to as an "IDCT unit") 207, and a color conversion unit 208. You.

The bit plane hierarchical decoding unit 206 performs a bit plane hierarchical decoding process on the hierarchically encoded data stored in the memory 205. IDCT unit 207
Performs an inverse DCT transform on the signal decoded by the bit plane hierarchical decoding unit 206 to return to a luminance / chrominance signal. The color conversion unit 208 converts the luminance / color difference signal supplied from the IDCT unit 209 into an image signal (RGB signal) S22.
And output.

The monitoring unit 115 shown in FIG. 1 includes the code amount monitoring unit 209 and the tile attribute determination unit 210. The code amount monitoring unit 209 performs the encoding process by the bit plane layer encoding unit 203 and the selector 204.
Monitor the amount of hierarchically encoded data stored in the memory 205 via the. Based on the monitoring result, the code amount monitoring unit 2
09 is the selector 20 via the tile attribute determination unit 210.
4 is appropriately controlled to control the amount of hierarchically encoded data stored in the memory 205.

At this time, the tile attribute discriminating unit 210 determines whether the tile image related to the encoded data stored in the memory 205 is a tile image that remains up to a high hierarchical level (encodes at a low compression rate) as shown in FIG. Determine whether the tile image is reduced to a lower number of layers (encoded with a higher compression rate),
The amount of hierarchically encoded data stored in the memory 205 is controlled. Although the tile attribute determination unit 210 is provided in the monitoring unit 115 shown in FIG. 1, the tile attribute determination unit 210 may be provided in the image area flag decoding unit 108 instead of the monitoring unit 115.

FIG. 3 is a diagram for explaining a method of switching the hierarchical level of hierarchical coding in a tile image composed of 32 × 32 pixels. In FIG. 3, tile images G1 to G5 and G8 (white tiles) indicate tile images using hierarchical coding up to a low hierarchical number, and the tile images G
6, G7, G9 to G16 (shaded tiles) indicate tile images using hierarchical coding up to a high hierarchical number.
When encoding this image, the tile attribute determination unit 210 illustrated in FIG. 2 gives priority to the above-described layers of the tile images G1 to G5 and G8 with a low number of layers according to the code amount generated by the encoding process. The selector 204 is controlled so as to delete it, and the amount of code stored in the memory 205 is reduced.

FIG. 4 shows the code amount monitoring unit 209 shown in FIG.
5 is a functional block diagram for realizing the operation of a tile attribute determination unit 210. FIG. 4, reference numeral 401 denotes a DCT
The DCT unit may be the DCT unit 202 shown in FIG. 2 or a DCT unit provided separately therefrom. 40
2a and 402b are inverse quantizers, 403a and 403b are I
The DCT units 404a and 404b provide a mean least square error (M
SE) calculation units, 405a and 405b are variable length code amount calculation units, and 406 is a determination unit. S41 is a target image quality instruction signal, and S42 is a target code amount instruction signal, which is input from a setting unit (not shown). S43 is an image signal, and S44 is a hierarchical level instruction signal.

Here, in FIG. 4, the hierarchy level A is
This is a hierarchy level higher than the hierarchy level B. For example, the hierarchical level A corresponds to a DCT coefficient in which the lower 4 bits are omitted, that is, linear quantization 1/16 (not shown).
The hierarchical level B corresponds to a DCT coefficient in which the lower one bit is omitted, that is, linear quantization 1/2 (not shown).

The DCT unit 402 receives the supplied image signal S
42 is DCT transformed and quantized, and the resulting D
Output CT coefficients. At this time, the DCT unit 402
Except for the lower 4 bits of the CT coefficient, that is, the DCT coefficient at the hierarchical level A is output to the inverse quantization unit 402a and the variable length code amount calculation unit 405a, and the inverse quantization unit 402b and the variable length code amount calculation unit 405b output the DCT coefficient. Outputs the DCT coefficient excluding the lower one bit of the DCT coefficient, that is, the DCT coefficient at the hierarchical level B.

Next, the variable length code amount calculation units 405a, 40
5b calculates the code amount at each hierarchical level based on the DCT coefficient supplied from the DCT section 402.
Also, the DC supplied to the inverse quantization units 402a and 402b
The T coefficient is inversely quantized by the inverse quantizers 402a and 402b, respectively, and further subjected to inverse DCT transform by the IDCT units 403a and 403b to be decoded. ID
The decoded images decoded by the CT units 403a and 403b are supplied to mean least square error (MSE) calculation units 404a and 404b, and the mean least square error calculation units 404a and 404b
And the mean least square error (MSE) of the difference between the decoded image and the original image related to the image signal S42 is the hierarchical level A or B, respectively.
Is calculated for

The determining unit 406 determines the hierarchical levels A and B calculated by the variable length code amount calculating units 405a and 405b.
, And the mean least square error (MSE) calculated by the above-mentioned mean least square error (MSE) calculation units 404a and 404b, a curve as shown in FIGS. The layer level is determined and output according to the instruction signal S41 or the target code amount instruction signal S42. Thus, at all hierarchical levels, the code amount and the mean least square error (MSE) of the difference between the decoded image and the original image are obtained.
, It is possible to determine an appropriate hierarchical level with a small amount of arithmetic processing.

FIG. 5 is a diagram showing an example of the relationship between the code amount and the mean least square error (MSE) of the difference between the decoded image and the original image. In FIG. 5, the horizontal axis is the code amount, and the vertical axis is the mean least square error. Reference numeral 501 denotes a point corresponding to the code amount at the hierarchical level A and the mean least square error (MSE), and reference numeral 502 denotes a point corresponding to the code amount at the hierarchical level B and the mean least square error (MSE). From these two points of data, the determination unit 406 shown in FIG. 4 calculates a parameter of a predetermined approximate curve to obtain a curve indicating the relationship between the code amount and the mean least square error (MSE). Then, the required code amount can be obtained from the target image quality indicated by the target image quality instruction signal S41. Further, the image quality can be estimated from the generated code amount.

FIG. 6 is a diagram showing an example of the relationship between the hierarchy level and the code amount. In FIG. 6, the horizontal axis is the hierarchical level, and the vertical axis is the code amount. Reference numeral 601 indicates a code amount at the hierarchical level A, and reference numeral 602 indicates a code amount at the hierarchical level B. From these two points of data, the determination unit 406 shown in FIG. 4 calculates a parameter of a predetermined approximate curve to obtain a curve indicating the relationship between the hierarchical level and the code amount. Thereby, the hierarchical level according to the target code amount indicated by the target code amount instruction signal is predicted.

FIG. 7 is a block diagram for explaining the process of determining the hierarchical level when the tile image is hierarchically encoded from the curves shown in FIG. 5 and FIG. . 7, reference numeral 701 denotes a code amount monitoring unit, 702 denotes a first target image quality setting unit, and 703 denotes a second target image quality setting unit.
704 is a target image quality setting unit, and 705 is a target code amount setting unit.
Is a hierarchy level determination unit. Here, the first target image quality setting unit 702 is for setting the image quality of the entire image, and the second target image quality setting unit 703, the target code amount setting unit 704, and the hierarchy level determination unit 705 are respectively This is for a tile image. The values set in the first and second target image quality setting units 702 and 703 are set by the target image quality instruction signal S41 shown in FIG. 4, and the value set in the target code amount setting unit 704 is the target code. It is set by the quantity indication signal S42.

When the target code amount is adjusted while monitoring the code amount by the code amount monitoring unit 701, the target code amount for each tile is determined and set in the target code amount setting unit 704, and the hierarchy level determination unit 705. Determines the hierarchy level for each tile. On the other hand, when the code amount is monitored by the code amount monitor unit 701, for example, when it is predicted that the code amount exceeds a preset code amount, the new target image quality is set to the first target image quality setting unit 702. Set. Then, the second target image quality setting unit 703 determines whether the image is a high image quality area (tiles to be left up to a high level) or a low image quality area (tiles of a low level) based on an image area flag for each tile. By setting the target code amount for each tile in the target code amount setting unit 704 based on the determination result, the layer level determination unit 605 determines the layer level for each tile.

FIG. 8 is a diagram for explaining an example of a hierarchical level. 8, reference numeral 81 denotes a bit plane of the least significant bit (LSB) of the DCT coefficient, and reference numeral 82 denotes a DC plane.
This is the second lower-order bit plane of the T coefficient. 8
Reference numerals 3, 84, 85, 86, and 87 are bit planes of the third, fourth, fifth, sixth, and seventh bits of the DCT coefficient, respectively, and reference numeral 88 is a bit plane of the most significant bit (MSB) of the DCT coefficient.

The data at the hierarchical level 1 is the coded data of the part except the lower 4 bits of the coefficient value of the DCT coefficient, that is, bit planes 88, 87, 86, and 85, and is realized by sending this data. Then, the layer level 2 sends the encoded data of the next bit plane 84.
Similarly, hierarchy level 3 is the next bit plane 83
, And the hierarchical level 4 sends the coded data of the bit plane 82.

When the bit plane 81 of the least significant bit (LSB) of the hierarchical level 5 at the final stage is transmitted, all the DCT coefficient values can be transmitted, and a reversible DC capable of reversible transformation is transmitted.
In the case of T, reversibility of the image can be realized at the hierarchical level 5 in the final stage. In this way, hierarchical coding is realized by coding the DCT coefficients for each bit plane.

FIG. 9 is a diagram showing an example of hierarchical coding, showing a tile image before code amount control, a tile image after code amount control, and a hierarchical level. FIG. 9 (1)
The tile image and the hierarchy level before code amount control are shown, and all the tile images have five layers. FIG. 9 (2)
The tile image after code amount control and the hierarchical level are shown. As a result of the code amount control, tile images having 3, 4, and 5 layers are mixed.

In the present embodiment, the encoding process in the color image encoding 104 shown in FIG.
The T-transform or the irreversible DCT transform may be used. However, by transmitting the bit planes at all hierarchical levels as described above, the image can be faithfully decoded when the reversible DCT transform is used. Can be. Hereinafter, the reversible DCT transform will be described.

FIG. 10 is a diagram for explaining the reversible DCT method. As shown in FIG. 10, the 8-point DCT transform is 2
It can be decomposed into point transformation and four-point transformation. 70a ~
70 g is a two-point conversion using coefficients a and -a,
Is a two-point conversion using coefficients b and c, and 72 is a four-point conversion. Hereinafter, the two-point conversion and the four-point conversion will be described.

FIG. 11 is a diagram showing an example of two-point reversible conversion. In FIG. 11, 1101 to 1103, 1107 to
1109 is a multiplier, 1104 to 1106 are adders, and 1110 to 1112 are subtractors. Multiplier 110
1 to 1103 and 2 by adders 1104 to 1106
A point conversion circuit is configured. First, the adder 1104
Performs addition operation between the input signal X _b a and k ₀ times by the multiplier 1101 (taking the 0.5 added floor function) value rounded off with the input signal X _a, the signal θ is the result Output _buff . The adder 1105 outputs the signal θ _buff output from the adder 1104 to the multiplier 11
02 by adding the k ₁ factor and signals inputted to the value obtained by rounding off to X _b, and outputs the a sum theta _a. Further, the adder 1106 generates a value obtained by multiplying the signal θ _a output from the adder 1105 by k ₂ by the multiplier 1103 and rounding it, and the signal θ output from the adder 1104.
adding the _buff, and outputs a signal theta _b is the addition result.
In this manner, the input signal X _a, X _b signal subjected to the two-point conversion from theta _a, theta _b is generated and outputted.

Further, multipliers 1107 to 1109 and subtracters 1110 to 1112 constitute a two-point inversion circuit. First, the subtractor 1110 outputs the signal θ _a to the multiplier 11
07 by subtracting the value obtained by rounding off to ₂ times k from the signal theta _b, and outputs a signal theta _{buff 'of} the result. The subtractor 1111 outputs the signal θ output from the subtractor 1110.
and it outputs a signal X _b by subtracting the value obtained by rounding off to ₁ × k by the multiplier 1108 to _{buff 'from} the signal theta _a.
Similarly, the subtractor 1112 subtracts a value obtained by multiplying the signal _Xb output from the subtractor 1111 by k ₀ by the multiplier 1109 and rounding it off from the signal θ _{buff ′} output from the subtractor 1110. and it outputs a signal X _a by. As described above, two-point conversion and two-point inverse conversion are performed.
The two-point transformation and the two-point inverse transformation described above have reversibility since rounding and addition / subtraction operations are performed symmetrically.

FIG. 12 is a diagram showing an example of the four-point reversible conversion. As shown in FIG. 12, input signals _{Xa '} , _Xb' , _{Xc '} ,
Signals θ _{a ′} , θ _{b ′} , θ _{c ′} , θ _{d ′ obtained} by performing a four-point conversion from X _{d ′}
, C15, multiplying by the coefficients c1, c2,..., C15 and adding up the rounded value realizes the four-point conversion. Further, since the circuit for performing the above four-point conversion is composed of a multiplier for multiplying and rounding off and an adder, 2
Similarly to the point conversion, a circuit for performing an inverse conversion from the multiplier and the subtractor can be configured, and a four-point reversible conversion can be realized.

The multiplication coefficients k ₀ , k ₁ , k ₂ , c ₁ , c ₂ ,..., C 15 in FIGS. 11 and 12 are the coefficients a, b, c, d, e, f, g shown in FIG. Can be obtained by the operation of the determinant. For example, coefficient a, -a 2
The multiplication coefficients k0, k1, and k2 for the point transformation are-
0.4142, 0.70711, -0.41421, and the multiplication coefficient k ₀ for the two-point conversion of the coefficients b and c,
k _1, k _2, respectively -0.66818,0.9238
8, -0.66818. Further, for example, the multiplication coefficients c1, c2,. . . , C15, c1 = −0.5941
7, c2 = −4.025543, c3 = −1.4351
3, c4 = 3.18456, c5 = 1.133324, c
6 = 0.69352, c7 = 0.21137, c8 =
0.44133, c9 = 0.21138, c10 =-
0.693512, c11 = -1.13326, c12
= 3.18454, c13 = 1.43517, c14 =
-4.002545, c15 = 0.594169.

As described above in detail, according to the present embodiment, the variable length code amount calculation unit 4 in the code amount monitoring unit 209
At steps 05a and 405b, the code amounts after the encoding processing at the hierarchical levels A and B are calculated, respectively. Based on the calculated code amounts after the encoding processing, the determination unit 406 calculates an approximate curve indicating the relationship between the hierarchical level and the code amount. Ask. Then, based on the obtained approximate curve, a hierarchical level corresponding to a target code amount is predicted and determined, and based on the prediction, the color image coding unit 104 outputs coded data that has been coded.

Thus, it is possible to predict and determine the hierarchical level at which the code amount after the encoding process becomes the target code amount in one encoding process without performing the encoding process a plurality of times. The code amount after the encoding process by the color image encoding unit 104 can be controlled. In addition, since the above-mentioned approximate curve is obtained from the code amount after the encoding process at the two hierarchical levels, the code amount after the encoding process at each hierarchical level is predicted with a small amount of arithmetic processing, and the appropriate hierarchical level is calculated. Can be determined.

Further, in the present embodiment, the image data to be coded is divided into a plurality of tile images having a predetermined size, and the hierarchical coding process is performed for each tile image. It is possible to control the code amount after the encoding process by the encoding unit 104 with high accuracy.

In the present embodiment, the mean least squares of the image related to the image data before the encoding process and the image related to the decoded image data after the encoding process at the hierarchical levels A and B are performed. The error is calculated by the mean least square error calculation unit 404
The determination unit 406 obtains an approximate curve indicating the relationship between the code amount and the average least square error from the calculated average least square error calculated using the values a and 404b. Based on the approximation curve, the encoding amount after the encoding process for obtaining the target image quality is predicted, so that the image related to the image data before the encoding process and the image data decoded after the encoding process The encoding process can be performed by controlling the hierarchical level so that the error from the image according to (1) is smaller than a predetermined value.

Further, since the code amount after the encoding process by the color image encoding unit 104 is controlled only at the hierarchical level, compatibility with the standard system can be maintained.

(Second Embodiment) In the second embodiment, the encoding amount is controlled by dividing the image into tile images of a predetermined size and controlling the hierarchical level of the tile image in the first embodiment. In this case, the coding amount is controlled by controlling the quantization step. Note that the configuration of a digital imaging device to which the image processing device according to the second embodiment is applied is the same as that of the digital imaging device shown in FIG. However, in the first embodiment, the number of levels of the hierarchical coding is used as the coding parameter and the decoding parameter that are switched according to the image area flag. However, in the second embodiment, the coding parameter and the decoding parameter are used. Is used as the quantization matrix.

That is, in the first embodiment, the amount of code generated from the color image coding unit 104 is controlled by controlling the number of levels of hierarchical coding.
In the embodiment, the number of quantization steps is controlled to control the amount of code generated from the color image coding unit 104.

FIG. 13 is a diagram for explaining the processing of the color image encoding unit 104 and the color image decoding unit 109 in the second embodiment. In FIG. 13, blocks having the same functions as the blocks shown in FIG. 2 are denoted by the same reference numerals, and redundant description will be omitted. In addition, blocks that are not the same as those illustrated in FIG. 2 but have corresponding functions are denoted by the same reference numerals.

In FIG. 13, reference numeral 202 'denotes a reversible discrete cosine transform unit (hereinafter, referred to as a "reversible DCT unit").
~ Reversible DCT transformation as shown in FIG.
Reference numeral 1301 denotes a quantization unit that reduces the data amount by quantizing the DCT coefficient supplied from the reversible DCT unit 202 ′. Reference numeral 1302 denotes a variable-length coding (VLC) unit, and the quantization value supplied from the quantization unit 1301 is
A Huffman encoding process using the bias of the data appearance probability is performed to further reduce the data amount.

A variable length decoding (VLD) unit 1303 performs Huffman decoding. An inverse quantization unit 1304 performs an inverse quantization process on the code data Huffman-decoded by the variable length decoding unit 1303 and returns the code data to a DCT coefficient value.
A code amount monitoring unit 209 ′ monitors the code amount stored in the memory 205, and switches the quantization step to the quantization unit 1301 when it is predicted that the code amount exceeds a preset code amount. Instruct.

An image area flag encoding unit 1305 encodes and compresses the image area flag and stores it in the memory 215. Reference numeral 1306 denotes an image area flag decoding unit for each block, and an inverse quantization unit 1304 that switches a quantization matrix for inverse quantization processing according to the decoding result.
To instruct.

FIG. 14 is a diagram showing an example of area division in the second embodiment. As shown in FIG. 14, an image to be encoded is divided into a character area and a photographic image area other than the character area. For example, when the image data and the image area flag are supplied from the PDL rendering unit 113, the lossless encoding area and the irreversible encoding area correspond to the character area and the photographic image area, respectively. Further, for example, when image data and an image area flag are supplied from the image scanner unit 112, an area having a relatively small quantization step and a quantization step Are divided into a relatively large area. By supplying an image area flag corresponding to the above area division,
The amount of code is controlled by making the quantization step fine in the character area and making the quantization step coarse in the photographic image area.

FIG. 15 is a block diagram showing a tile image composed of 32 × 32 pixels, which is converted into blocks G1 ′ to G8 ′ for 8 × 8 DCT conversion.
G16 ', and a region G1' with a fine quantization step
G5 ', G8', region G with a large quantization step
It is the figure which showed the area | region divided into 6 ', G7', and G9'-G16 '. In FIG. 15, regions G1 'to G1
5 'and G8' are portions where image quality deterioration is relatively conspicuous, and regions G6 ', G7' and G9 'to G16' are portions where image quality deterioration is not conspicuous.

FIG. 16 is a diagram showing an example of a basic quantization matrix for 8 × 8 DCT coefficients. FIG.
(1) is an example of a quantization matrix used for an area to be irreversibly quantized. FIG. 16B shows a quantization matrix in which all components of the quantization matrix used for the region to be reversibly quantized are one, and if this matrix is used, no quantization is performed.

As described above, an image to be coded is divided into a region in which image quality degradation is relatively conspicuous and a region in which image quality degradation is less conspicuous, and an image area flag corresponding to the area is divided into image area flags. This is supplied to the encoding unit 103. Based on the supplied image area flag, the image area flag encoding unit 103
Instructs the quantization unit 1301 in the color image encoding unit 104 to switch the quantization step.

In accordance with the above-mentioned instruction, the quantization unit 1301 performs the irreversible quantization processing shown in FIG. The input signal is quantized using a quantization matrix to reduce the amount of data. On the other hand, when the quantization unit 1301 is instructed to reduce the quantization step because image quality degradation is conspicuous,
The input signal is quantized by using a quantization matrix such as a reversible quantization process shown in 6 (2). In this way, the code amount stored in the memory 205 is controlled.

When performing the encoding process as described above, if the code amount monitoring unit 209 'predicts that the code amount stored in the memory 205 will exceed a preset code amount, the code amount monitoring unit 209' The unit 1301 is instructed to switch the quantization step. According to the instruction, the quantization unit 1301 changes the quantization step to a quantization step using the above-described quantization matrix for irreversible quantization processing, or deletes high-layer (high-resolution) data in the case of hierarchical encoding processing. For example, the code amount is reduced by controlling the hierarchical level, and the code amount stored in the memory 205 is controlled.

As described above, according to the second embodiment, the image data to be encoded is divided into a character area and a photographic image area, and according to the image area flag for identifying the image area. When the image to be coded is a character area, lossless coding is performed, and when the image to be coded is a photographic image area, lossy coding is performed.

As a result, it is possible to increase the code amount in the region where the image quality degradation is conspicuous, and to reduce the code amount in the region where the image quality degradation is not conspicuous. An encoding process can be performed.

When the code amount monitoring unit 209 ′ predicts that the code amount after encoding is larger than a predetermined code amount, the quantization unit 1301 irreversibly quantizes the quantization matrix used in the quantization unit 1301. When switching to the matrix, the lossless encoding process can be switched to the irreversible encoding process, and the code amount after the encoding process can be reduced.

Similarly, when the hierarchical coding process is used, when the code amount monitoring unit 209 'predicts that the code amount after the encoding is larger than the predetermined code amount, the image data is encoded. By controlling the number of layers to be encoded, the amount of code after encoding can be reduced.

(Other Embodiments of the Present Invention) In order to operate various devices to realize the functions of the above-described embodiments, an apparatus connected to the various devices or a computer in a system is connected to the above-described embodiment. Supply the software program code to realize the functions of
The present invention also includes those implemented by operating the various devices according to a program stored in a computer (CPU or MPU) of the system or apparatus.

In this case, the program code of the software implements the functions of the above-described embodiment, and the program code itself and means for supplying the program code to a computer are provided.
For example, a recording medium storing such a program code constitutes the present invention. As a recording medium for storing such a program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, C
D-ROM, magnetic tape, nonvolatile memory card, R
OM or the like can be used.

The computer executes the supplied program code, not only to realize the functions of the above-described embodiment, but also to execute the program code on an operating system (OS) or another operating system running on the computer. Needless to say, even when the functions of the above-described embodiments are realized in cooperation with application software and the like, such program codes are included in the embodiments of the present invention.

Further, after the supplied program code is stored in a memory provided in a function expansion board of a computer or a function expansion unit connected to the computer, the function expansion board or the function expansion unit is specified based on the instruction of the program code. It is needless to say that the present invention also includes a case where the CPU or the like provided in the first embodiment performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0093]

As described above, according to the present invention, the code amount after hierarchical coding of image data is calculated for each of a plurality of layers, and the calculated code amount after hierarchical coding is calculated based on the calculated code amount after hierarchical coding. Since the amount of code after the hierarchical encoding process according to the number is predicted and the hierarchical encoding process is performed according to the result, the encoding process is performed a plurality of times, or the encoded image data is decoded and further encoded. It is possible to easily control the encoding amount after the hierarchical encoding processing by one encoding processing without performing complicated processing such as conversion into a single layer.

Further, an approximate curve indicating the relationship between the number of layers and the code amount after the hierarchical coding process is obtained from the calculated code amount after the hierarchical coding process, and the approximated curve after the hierarchical coding process according to the number of layers is obtained. When the code amount is predicted, the code amount after the hierarchical coding process can be controlled by controlling the number of layers for coding the image data according to the prediction result.

Further, a least square error between the image related to the image data and the respective decoded images related to the image data after the hierarchical coding process in a plurality of layers is calculated, and based on the calculated least square error, When an approximate curve indicating the relationship between the number of layers and the least square error is obtained, and the amount of code after the hierarchical coding processing for obtaining a target image quality is predicted, the image related to the image data and the hierarchical coding The image data can be encoded by controlling the number of layers to be encoded so that the least square error of the processed image data with respect to the decoded image is smaller than a predetermined value.

When the image data is divided into regions and the encoding process is performed in accordance with an image region flag indicating the attribute, regions where image quality deterioration is conspicuous and regions where image quality deterioration is not conspicuous are distinguished from each other. Encoding processing can be performed to obtain an appropriate code amount.

When the code amount after the hierarchical coding process is predicted to be larger than the predetermined code amount, when the lossless coding process is switched to the irreversible coding process,
The code amount can be reduced. Similarly, when the code amount after the hierarchical encoding process is predicted to be larger than the predetermined code amount, the number of layers for encoding the image data is controlled. By removing the data, the code amount can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a digital imaging device to which an image processing device according to a first embodiment is applied.

FIG. 2 is a diagram for explaining processing of a color image encoding unit 104 and a color image decoding unit 109;

FIG. 3 is a diagram for explaining a method of switching a hierarchical level of hierarchical coding.

FIG. 4 shows a code amount monitoring unit 209 and a tile attribute determination unit 2
It is a functional block diagram for realizing operation | movement of No.10.

FIG. 5 is a diagram illustrating an example of a relationship between a code amount and a mean least square error (MSE) of a difference between a decoded image and an original image.

FIG. 6 is a diagram illustrating an example of a relationship between a hierarchy level and a code amount.

FIG. 7 is a block diagram for explaining a process of determining a hierarchical level when hierarchically encoding a tile image.

FIG. 8 is a diagram illustrating an example of a hierarchy level.

FIG. 9 is a diagram illustrating an example of hierarchical coding.

FIG. 10 is a diagram for explaining a reversible DCT method.

FIG. 11 is a diagram illustrating an example of two-point reversible conversion.

FIG. 12 is a diagram illustrating an example of four-point reversible conversion.

FIG. 13 is a diagram for describing processing of a color image encoding unit 104 and a color image decoding unit 109 according to the second embodiment.

FIG. 14 is a diagram illustrating an example of area division according to the second embodiment.

FIG. 15 is a diagram showing another example of area division.

FIG. 16 is a diagram illustrating an example of a basic quantization matrix for 8 × 8 DCT coefficients.

[Explanation of symbols]

 Reference Signs List 101 image input unit 102 compression block buffer 103 image area flag encoding unit 104 color image encoding unit 105 compression memory 106 external storage device 107 decoding memory 108 image area flag decoding unit 109 color image decoding unit 110 expansion block buffer 111 interface 112 image scanner unit 113 page description language rendering unit 114 selector 115 monitoring unit 201 color conversion unit 202 discrete cosine conversion unit 203 bit plane layer coding unit 204 selector 205 memory 206 bit plane layer decoding unit 207 inverse discrete cosine conversion unit 208 color conversion Unit 209 code amount monitoring unit 210 tile attribute determination unit

 ────────────────────────────────────────────────── ─── Continued on the front page F term (reference) 5C059 KK01 KK22 MA00 MA23 MA35 MA45 MC14 ME02 ME05 PP16 PP20 SS11 SS20 SS28 TA39 TA46 TA53 TC18 TD01 TD06 UA02 UA31 5C078 BA57 BA58 CA02 DA07 DB01 DB11 DB19

Claims (20)

[Claims] 1. An image processing apparatus having a function of performing hierarchical coding processing of image data by bit-plane coding, the code amount calculating a code amount of the image data after the hierarchical coding processing in each of a plurality of layers. Calculating means, based on the code amount calculated by the code amount calculating means,
Code amount prediction means for predicting the code amount after hierarchical coding processing according to the number of layers, and coding means for performing hierarchical coding processing on the image data according to the prediction result by the code amount prediction means An image processing apparatus characterized by the above-mentioned. 2. The method according to claim 1, wherein the code amount predicting unit calculates an approximate curve indicating a relationship between the number of layers and the code amount after the hierarchical coding process based on the code amount calculated by the code amount calculating unit. 2. The apparatus according to claim 1, further comprising: an approximate curve calculation unit, wherein a code amount after the hierarchical encoding process according to the number of layers is predicted based on the approximate curve obtained by the first approximate curve arithmetic unit. 3. The image processing apparatus according to any one of the preceding claims. 3. The code amount predicting unit includes: an error calculating unit configured to calculate a least square error between an image related to the image data and each decoded image related to image data after hierarchical coding processing in a plurality of layers. 3. The method according to claim 1, further comprising: predicting a code amount of the image data after the hierarchical coding process for obtaining a target image quality, based on the least square error calculated by the error calculation unit. The image processing apparatus according to any one of the preceding claims. 4. The second approximation curve calculation for obtaining an approximation curve indicating the relationship between the number of layers and the least square error based on the least square error calculated by the error calculation means. Means for predicting a code amount after hierarchical coding of the image data for obtaining a target image quality based on the approximate curve obtained by the second approximate curve calculation means. Item 4. The image processing device according to item 3. 5. The image processing apparatus according to claim 1, wherein the encoding unit divides the image data into a plurality of tile images and performs a hierarchical encoding process on each of the plurality of tile images. 6. The image processing apparatus according to claim 1, wherein said encoding means performs a hierarchical encoding process in accordance with an image area flag indicating an attribute of the area-divided image data. The image processing device according to claim 1. 7. The image area flag is a flag for identifying a character area and a photographic image area, and the encoding means performs a lossless encoding process when the image area flag is a character area flag. 7. The image processing apparatus according to claim 6, wherein the irreversible encoding process is performed when the image area flag is a photographic image area flag. 8. The method according to claim 7, wherein when the prediction result by said code amount predicting means is larger than a predetermined code amount, said coding means switches said lossless encoding processing to said irreversible encoding processing. An image processing apparatus according to claim 1. 9. An image processing apparatus having a function of hierarchically encoding image data by bit-plane encoding, wherein the image data is divided into a character area and a photographic image area, An image processing apparatus according to claim 1, wherein a lossless encoding process is performed for the character region and an irreversible encoding process is performed for the photographic image region in accordance with an image area flag for identifying the area. 10. An image processing method for hierarchically encoding image data by bit-plane encoding, wherein a code amount of the image data after hierarchical encoding is calculated for each of a plurality of layers, and the calculated hierarchical level is calculated. An image processing method comprising: predicting a code amount after hierarchical coding processing according to the number of layers based on a code amount after coding processing, and performing hierarchical coding processing on the image data according to the result. . 11. A method for determining an approximate curve indicating a relationship between the number of layers and the code amount after the hierarchical coding process, and predicting the code amount after the hierarchical coding process according to the number of layers. The image processing method according to claim 10. 12. Calculating a least square error between the image related to the image data and each decoded image related to the image data after the hierarchical coding process in a plurality of layers, and calculating a target value based on the calculated least square error. The image processing method according to claim 10 or 11, wherein a code amount of the image data after the hierarchical coding process for obtaining the image quality is predicted. 13. An image processing method comprising: obtaining an approximate curve indicating a relationship between the number of layers and the least square error, and predicting a code amount of the image data after the layer coding processing for obtaining a target image quality. 13. The image processing method according to claim 12, wherein: 14. The image processing method according to claim 10, wherein the image data is divided into a plurality of tile images, and each of the image data is subjected to hierarchical coding processing. 15. The image processing apparatus according to claim 10, wherein the image data is divided into regions, and hierarchical coding is performed according to an image region flag indicating an attribute of the region-divided image data. Image processing method. 16. The image area flag is a flag for identifying a character area and a photographic image area. When the image area flag is a character area flag, lossless encoding processing is performed. 16. The image processing method according to claim 15, wherein irreversible encoding processing is performed when the flag of the area is set. 17. The method according to claim 16, wherein when the code amount after the hierarchical coding process is predicted to be larger than a predetermined code amount, the lossless coding process is switched to the irreversible coding process. The image processing method described in the above. 18. An image processing method for hierarchically encoding image data by bit-plane encoding, wherein the image data is divided into a character area and a photographic image area, and the character area and the photographic image area are separated from each other. An image processing method comprising: performing lossless encoding processing in the case of the character area and performing irreversible encoding processing in the case of the photographic image area according to an image area flag to be identified. 19. A computer-readable recording medium on which a program for causing a computer to function as each of the means according to claim 1 is recorded. 20. A computer-readable recording medium on which a program for causing a computer to execute the processing procedure of the image processing method according to claim 10 is recorded.