JP2006262294A - Image processing apparatus, image processing method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for executing code quantity control after coding, with high accuracy, by taking into account the effects of the distortion amount due to image attributes on the image quality. <P>SOLUTION: The image processing apparatus sets the degree of importance (priority) on coded data in the unit of code sequences (packets) for configuring the coded data depending on a distortion amount, image attribute information, and distortion amount reference by each image attribute of a decoded and reproduced image of the coded data. Alternatively, the image processing apparatus places the degree of importance (priority) on coded data in units of code sequences (packets) for configuring the coded data, corresponding to an image partial region depending on the distortion amount, image attribute information, and distortion amount reference by each image attribute of a decoded and reproduced image of the coded data, corresponding to the image partial region. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium. Specifically, the present invention relates to a method for (re-) forming code data based on the importance (priority order) of a code string constituting the code data. . Particularly, the importance (priority order) of the code string is set based on the distortion amount and image attribute information of the decoded reproduction image of the code data, and the distortion amount standard for each image attribute, and the code data is set based on the set importance. Is selected (code amount control).

  In general, the importance of code data formed when advancing code amount control in the course of code processing is identified in units of partial code strings. However, in general, the importance calculation result for each partial code string in the code processing process is not reflected after the code processing.

  In general, the degree to which deterioration of the reproduction amount of the encoded data calculated in the encoding process is different depending on the image attribute, and the distortion amount does not reflect the sensitivity for each image attribute. Since the frequency sensitivity is reflected, it is not possible to calculate a suitable degree of importance in units of code strings only from the amount of distortion calculated in the encoding process.

On the other hand, as image compression methods, international standards such as JPEG and JPEG2000 are known. In the JPEG2000 (ISO / IEC 15444-1) standard encoding process, a high-accuracy code amount that takes into account the frequency sensitivity visual characteristic (VW information) information used in the code amount control at the time of encoding after encoding. Control is not possible.
In addition, it is impossible to control the code amount in consideration of the frequency sensitivity visual characteristics, the image attribute information for each image area, and the resolution characteristics of the output device at the time of encoding.

  Patent Document 1 discloses a technique for calculating the distortion amount of code data. In this patent document 1, as in JPEG2000, in an image processing method for compressing image information by truncation of a bit plane of image information, the bit plane is truncated based on the number of most significant bits (MSBs) in each bit plane. Truncation is performed by estimating the amount of distortion of the image at the time, but the function of controlling the amount of code after encoding is not mentioned.

  Further, in Patent Document 2, an image at the time of encoding can be obtained by making it possible to know the amount of distortion of an image when the image is progressively displayed using the remaining data obtained by removing an error portion from the code string. The configuration of the code data is adjusted after encoding using information relating to the amount of distortion.

  Further, in Patent Document 3, assuming that encoding by JPEG2000 is performed, the tile attribute is set to SNR (layer) progressive based on the distortion amount (error) of the image when the tile attribute is a character based on the tile attribute of the tile unit. In the case of a photograph, code data is formed by changing the progressive order to resolution progressive. By forming code data by SNR (layer) progressive, code amount control (rate control) based on the distortion amount (error) of an image can be performed after encoding. In general, the code data of each layer is composed of a plurality of code strings, and prioritization of importance cannot be performed in units of code strings, so that high-precision and fine code amount control cannot be performed.

Further, there are Patent Document 4 and Patent Document 5 as technologies related to priority setting in packet units. This Patent Document 4 has a table for setting priorities in units of packets, and packet transmission / reception is controlled based on priorities in units of packets.
In Patent Document 5, a priority field is provided for code data, but it is not intended for overlapping code data.

  As a technique related to code amount control in a typical JPEG2000, there is Patent Document 6. In Patent Document 6, code amount control is performed in units of tiles in encoding according to the JPEG2000 standard.

Patent Document 7 discloses a technique for editing code data with a typical JPEG2000 code string creation device. However, there is no packet to be deleted or rearranging means considering the image quality.
JP 2003-304405 A JP 2004-186861 A JP 2003-23544 A JP 2003-338840 A JP 2003-209839 A JP 2003-32680 A JP 2003-169333 A

  In the code processing of the JPEG2000 standard, code amount control based on code amount control information at the time of encoding (discrimination information related to frequency sensitivity characteristics of code data) cannot be performed in code data editing processing at a code level after encoding processing. I have a problem.

  By the way, the degree to which the deterioration of the distortion amount of the reproduced image of the code data generally varies depending on the image attribute, and although the distortion amount reflects the frequency sensitivity to the encoding error, it does not reflect the sensitivity for each image attribute. A suitable decoded image quality could not be obtained only from the amount of distortion.

  In addition, the JPEG2000 standard codes are configured as a set (order) of specified minimum unit code strings (referred to as packets), and can be edited in units of code strings at the code level. There is a demand to transfer more important code strings earlier for reasons such as error handling at the time of transfer of code data composed of a set of code strings (packets) such as JPEG2000 standard codes and convenience for decoding. .

For example, it can be realized by assigning priorities in units of code strings and transferring the code data according to such priorities. However, in the encoding of the JPEG2000 standard, the order of code strings that can be decoded is limited to several progressive orders, and is configured in a progressive order selected from these progressive order options.
However, since these progressive orders are not determined based on the priority order of each minimum unit packet (code string) constituting a code, they are defined in the JPEG2000 (J2K: ISO / IEC 15444-1) standard. In the order of the code sequence in the progressive order, the important code sequence cannot always be preferentially transferred first, or the relatively important code sequence is deleted and the unimportant code sequence is deleted at the time of code amount control. If not it was happening. Also, the reproduction image quality is controlled in units larger than the code string (packet), and relatively fine image quality control cannot be performed.

  The present invention has been made in consideration of the above situation, and image processing that performs code amount control after encoding with high accuracy in consideration of the influence of image distortion on the image quality. An object is to provide an apparatus, an image processing method, a program, and a recording medium.

  In order to solve the above-described problem, the image processing apparatus according to claim 1 is characterized in that the code data includes the distortion amount of the decoded reproduction image of the code data, the image attribute information, and the distortion amount criterion for each image attribute. It has the function to set importance (priority order) in the code string (packet) unit which constitutes.

  The invention of the image processing apparatus according to claim 2 corresponds to the image partial area by the distortion amount of the decoded reproduction image of the encoded data corresponding to the image partial area, the image attribute information, and the distortion amount standard for each image attribute. It has a function of setting importance (priority order) in units of code strings (packets) constituting code data to be performed.

  According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect, the order of the code strings constituting the code data is changed according to the importance (priority order) of the code string (packet) unit. It has a function of reconstructing code data.

  According to a fourth aspect of the present invention, there is provided the image processing apparatus according to the first or second aspect, wherein the code sequence of the code sequence constituting the code data is in code sequence order according to the importance (priority order) of the code sequence (packet) unit. It has the function to transmit.

  According to a fifth aspect of the present invention, in the image processing apparatus according to the fourth aspect of the present invention, depending on the importance level (priority order) of the code string (packet) unit, the code string unit constituting the code data corresponds to the transmission status. It has a function of transmitting a code string by cutting it halfway.

  According to a sixth aspect of the present invention, in the image processing apparatus according to the fourth or fifth aspect, the code data constituting the decoding target is configured by replacing a part of the code string constituting the code data with a dummy code string having a short length. It has the function to perform.

  A seventh aspect of the present invention is the image processing apparatus according to the third aspect, further comprising means for comparing a distortion amount for each image area with a distortion amount reference, and the code data corresponding to the area having the specific image attribute. It is characterized by having means for reconstructing the code data so that is equal to or greater than the distortion amount standard.

  According to an eighth aspect of the present invention, there is provided the image processing apparatus according to the third aspect, further comprising means for comparing a distortion amount for each image attribute with a distortion amount reference, and an image reproduced by the reconstructed code data. Is code data that is equal to or greater than a distortion amount standard for each image attribute of a specific region in all image attributes.

  According to a ninth aspect of the present invention, in the image processing apparatus according to the second aspect, the partial region of the image is a preset attention (ROI) region.

  According to a tenth aspect of the present invention, in the image processing device according to any one of the first to ninth aspects, the distortion amount of the decoded reproduction image of the code data according to the first or second aspect is calculated in the code data generation process. It is the amount of distortion made.

  According to an eleventh aspect of the present invention, in the image processing device according to the tenth aspect, in the code data generation process according to the first or second aspect, the low-pass filter and the high-pass filter are vertically aligned with respect to the input image. And subband generation means for generating subbands by applying in a horizontal direction, filtering means for hierarchically filtering the lowband component subbands, and subbands generated by the filtering means. A code block generating means for generating a code block of a predetermined size, a bit plane generating means for generating a bit plane from the most significant bit to the least significant bit for each code block, and an encoding pass for each bit plane Coding pass generation means for generating, arithmetic coding means for performing arithmetic coding in the coding pass, An arithmetic code distortion amount calculating means for calculating the distortion amount of the technical code, wherein the distortion amount of the decoded reproduction image of the code data described in claim 1 or 2 is calculated by the arithmetic code distortion amount calculation means It is characterized by being.

  According to a twelfth aspect of the present invention, in the image processing device according to the tenth aspect, in the code data generation process according to the first or second aspect, the code amount is reduced by truncation in units of bit planes for each image area. Encoding is performed, and the number of most significant bits in each bit plane for each image area is used as a distortion amount.

  A thirteenth aspect of the present invention is the image processing apparatus according to any one of the first to twelfth aspects, further comprising means for designating an image area and means for extracting code data in a range corresponding to the image area. It is characterized by that.

  According to a fourteenth aspect of the present invention, in the image processing apparatus according to the thirteenth aspect, the means for designating the image partial area and the importance of the code string (packet) constituting the code data corresponding to the image area. And means for reconstructing the code data.

  According to a fifteenth aspect of the present invention, in the image processing device according to any one of the first to fourteenth aspects, the code data is data encoded based on JPEG2000 (ISO / IEC 15444-1) standard. Features.

  According to a sixteenth aspect of the present invention, in the image processing apparatus according to the fifteenth aspect, the image partial area according to the second aspect is a JPEG2000 (ISO / IEC 15444-1) standard precinct unit or tile unit. Features.

  According to a seventeenth aspect of the present invention, in the image processing apparatus according to the fifteenth aspect, the code string unit according to the first or second aspect is a packet unit of JPEG2000 (ISO / IEC 15444-1) standard. Features.

  The image processing method according to claim 18 is a code string (packet) unit constituting the code data by a distortion amount of the decoded reproduction image of the code data, image attribute information, and a distortion amount reference for each image attribute. The importance level (priority order) is set by.

  The image processing method according to claim 19 corresponds to the image partial area based on the distortion amount and image attribute information of the decoded reproduction image of the encoded data corresponding to the image partial area, and the distortion amount criterion for each image attribute. The importance (priority order) is set for each code string (packet) constituting the code data to be processed.

The invention according to claim 20 is a program for causing a computer to realize the function of the image processing apparatus according to any one of claims 1 to 17.
A twenty-first aspect of the invention is a computer-readable recording medium on which the program of the twentieth aspect is recorded.

  According to the present invention, code amount control after encoding can be performed with high accuracy in consideration of the influence of the distortion amount due to the image attribute on the image quality.

  Hereinafter, preferred embodiments of an image processing apparatus of the present invention will be described with reference to the drawings.

<Basic concept of the present invention>

First, the basic concept of the present invention will be described.
As seen in the JPEG2000 (J2K: ISO / IEC 15444-1) standard, a code with a peripheral structure can be partially selected without decoding all of the code data according to the requirements of the output device. Or can be deleted, decoded, and reproduced.
The code data is composed of a set (order) of defined code strings, and is characterized in that it can be edited in units of code strings at the code level.

  The editing of the code string in the present invention is mainly performed by deleting a specific code, adjusting the code amount, or changing the order of the code string so as to transfer an important code string at an early stage. In this case, it is desirable that the code strings are appropriately arranged according to importance or priority. Otherwise, it may happen that more important codes are deleted and less important codes are not deleted. Alternatively, it may happen that more important codes are transferred later and less important codes are transferred first. Therefore, how to determine the priority order becomes a problem.

  By the way, as seen in the code of the JPEG2000 (J2K: ISO / IEC 15444-1) standard, when the code is hierarchically configured, editing in units of code strings at the code level functions effectively. In such hierarchical encoding, codes are hierarchically divided by a plurality of scales.

The inclusion of a plurality of scales is convenient for constructing code data by changing the scale according to the purpose and changing the priority of the code string unit described above. In other words, since the reference is single, it is convenient to select one scale and arrange it in one dimension.
However, it is difficult to comprehensively evaluate a plurality of scales and arrange them in one dimension. This is because the reference is multi-dimensional and cannot be simply compared.

The entire image processing of the present invention is processed in the following procedure.
(1) Input image data.
(2) The distortion amount is calculated for each code string (packet) constituting the code data, and the encoding process is performed.
(3) The priority or importance of each code packet is set using the distortion amount criterion for each image attribute.
(4) Utilizing the set priority order or importance of the code packet unit, code sequence order control, transmission code stop control, and code amount control are performed.
(5) Perform decryption processing.
(6) Output image data.

  As described above, the feature of the present invention is that, as described above, a distortion amount is calculated for each code string (packet) constituting code data at the time of encoding processing, and encoding is performed using a distortion amount criterion for each image attribute after encoding. The packet unit priority (importance) is set and used for code sequence control, transmission code abort control, and code amount control.

  Hereinafter, coding based on the specification of the JPEG2000 standard (ISO / IEC 154444-1) is taken as an example. First, calculation of distortion of a code string in a partial code string unit in the coding process will be described, and at the code level A function for calculating importance or priority in units of code strings will be described.

  Codes based on the JPEG2000 standard are hierarchical codes and are divided by unit codes (code strings). Therefore, rearrangement of code string units, deletion of specific code strings, addition of codes, Since it is easy to edit (parse) the code data at the column level, it is possible to easily access the code data and control the code amount based on the importance in the present invention by targeting the encoded data based on the JPEG2000 standard. it can.

<Control method for setting importance or priority for each encoded code string>

Next, an importance or priority setting method in units of code strings after encoding based on the JPEG2000 standard will be described.
In the image processing apparatus according to the embodiment of the present invention, priority is given to each code string constituting the code data based on the image attribute information in the specific area, the calculation of the distortion amount in the specified area, and the distortion amount criterion for each image attribute. By having the function of setting the order, the code data is deleted in units of code strings, and some unimportant code strings (packets) are deleted, and the code amount and image quality are adjusted.

  FIG. 1 is a block diagram showing a configuration of importance setting in the image processing apparatus according to the embodiment of the present invention. 1, the image processing apparatus includes an input processing unit 10, an encoding processing unit 20, a code string unit importance calculation unit 30, a code data editing unit 40, a decoding processing unit 50, an output processing unit 60, and an image data storage unit 1. (70), a code data storage unit 80, and an image data storage unit 2 (90).

  The image data input from the input processing unit 10 is stored in the image data storage unit 1 (70), and the data encoded by the encoding processing unit 20 is stored in the code data storage unit 80. The stored code data is transferred to the decoding side via a network or the like. The stored code data is decoded by the decoding processing unit 50, the image data is reproduced, stored in the image data storage unit 2 (90), and the image data reproduced by the output processing unit 60 is output.

  Further, the code data editing unit 40 converts the code data stored in the code data storage unit 80 on the basis of the importance of the code string unit constituting the code data set by the code string unit importance calculation unit 30. The code data is reconstructed (code order control, code amount control, etc.).

  The code string unit importance calculation unit 30 includes a code string unit attribute information extraction unit 31, a code string unit distortion amount extraction unit 32, and a code string unit importance setting unit 33.

As will be described later, the code string unit attribute information extraction unit 31 uses the image attribute information stored in the image attribute information storage unit 35 in advance or calculated in the encoding process, as described later. In the conversion process, attribute information is extracted for each code string.
As will be described later, the code string unit distortion amount extraction unit 32 extracts the distortion amount for each code string in the encoding process.
The code string unit importance level setting unit 33 includes the distortion amount reference data for each image attribute stored in the image attribute distortion amount reference storage unit 34, the distortion amount for each code string extracted by the code string unit distortion amount extraction unit 32, and The importance level of the code string unit is set using the attribute information of the code string unit extracted by the code string unit attribute information extracting unit 31.

(1) Code stream reconstruction

  In the present invention, the importance of each code string (packet) constituting the code data is calculated based on the image attribute information in the specific area, the distortion amount in the specified area, and the distortion amount criterion for each image attribute. Are rearranged in order from the code sequence having the highest importance to reconstruct the code data. As a result, it is possible to provide means for reconstructing the code data in units of code strings in accordance with the importance.

  In addition, since the reconstructed code data is transferred, packets can be transferred in an order other than the progression order defined by the code data specifications in the decoding process. It is possible to provide a means for preferentially transferring.

  Also, the importance or priority of each code string (packet) constituting the code data is calculated, the code strings are rearranged in order of importance or priority, the code data is reconfigured, and the reconfigured code When data is sequentially transferred in units of code strings, if the code string is terminated in the middle according to the transfer status in units of code strings constituting the code data, the transfer code data is reconfigured according to the transfer status. Therefore, it is possible to provide a function of preferentially transmitting an important code string.

  At this time, a part of the code string constituting the code data is replaced with a dummy code string having a short length to form the code data to be decoded, and matching is performed so that the code data condition is satisfied after the code data is transmitted. Ensure consistency with the rules of decryption processing. That is, when a part of the code string (packet) is deleted and the progression order condition defined by JPEG2000 is not satisfied, the progression order condition defined by JPEG2000 is satisfied.

(2) A distortion amount control method based on a distortion amount standard for each image area:

  In the present invention, the distortion amount for each image area is compared by comparing the distortion amount for each image area with the distortion amount standard so that the code data corresponding to the specific image area is equal to or greater than the distortion amount reference. Avoid distortion above the quantity standard.

  Also, a code string (packet) constituting code data corresponding to the image partial area by the distortion amount of the decoded reproduction image of the code data corresponding to the image partial area, the image attribute information and the distortion amount standard for each image attribute By setting the importance (priority order) in units, it is possible to provide means for adjusting the image quality by deleting or adding a part of the unimportant code string (packet) of the code data in the designated area.

Providing means for controlling the amount of code for each image area based on an important area by setting the specific area of the image as a target ROI area set in advance (an area to be subjected to high image quality processing) Can do. For example, the reproduction quality of important character information is improved by using the specific image attribute as character information.
It can also be used to calculate a rate within an allowable deterioration range without image quality deterioration.

<Application system using importance for each code string>

(1) The code data on the server is accessed from the client side using the importance.

Next, a configuration in which a part of code data is accessed from the code string on the server from the client side using the importance will be described.
FIG. 2 is a block diagram showing the configuration of the image processing system according to the embodiment of the present invention. In the figure, in the image processing system, a server 100 and one or more clients 200 are connected via a network.

  The server 100 includes an image region designation unit 110, a region range corresponding code data selection unit 120, a designated region image attribute information extraction unit 130, a designated region distortion amount calculation unit 140, a code string selection unit 150, a code stream generation unit 170, An area unit image attribute information storage unit 135, an area unit distortion information storage unit 145, a code data storage unit 155, and a distortion amount reference storage unit 165 for each image attribute are included.

The image area designating unit 110 designates the requested image area based on the code stream upload for each client 200.
The area unit distortion information storage unit 145 stores area unit distortion information extracted at the stage where code data is generated by the process described above.

The area range corresponding code data selection unit 120 selects area range corresponding code data based on the requested image area designation.
The designated region image attribute information extraction unit 130 extracts image attribute information in the designated region from the region unit image attribute information storage unit 135 based on the selected region range corresponding code data.
The designated area distortion amount calculation unit 140 extracts area unit distortion information for the selected area range corresponding code data from the area unit distortion information storage unit 145 and calculates the distortion amount in the designated area.

The code string selection unit 150 includes the extracted image attribute information in the designated area, the calculated distortion amount in the designated area, and the distortion for each image attribute stored in the distortion amount reference storage unit 165 for each image attribute. A code string that constitutes code data of a region designated based on the quantity criterion is selected from the code data storage unit 155.
The code stream generation unit 170 configures a code stream with the selected code string, and transmits the code stream to the client 200 side.

  With this configuration, a part of the code data can be accessed based on the distortion amount of the code data in units of packets. In this case, a means for accessing the image area can be provided by designating the image area and extracting code data in a range corresponding to the image area. Furthermore, if the code data is extracted based on the importance of the code string (packet) constituting the code data in the range corresponding to the image area, the code data can be accessed based on the importance of the code string. .

(2) Code amount adjustment using importance

As described above, it is possible to analyze the header of code data generated in compliance with the JPEG2000 specification, and to truncate (code string control) or edit the code data in tile (image area) units or precinct units.
In the present invention, the code amount can be adjusted in units of code strings (packets) using the importance set in units of code strings constituting the code data.
For example, by setting the code data corresponding to the specific image area so that the code data is equal to or higher than the distortion amount standard, it is possible to configure so that distortion exceeding the distortion amount standard for each image area does not occur.

  Also, by reconfiguring the code data so that the code data corresponding to the area having the specific image attribute is equal to or greater than the distortion amount standard, the code data is partially deleted in units of code strings and some unimportant code strings (packets) are deleted. Alternatively, it is possible to adjust the code amount and to guarantee the image quality for each image attribute. In addition, since code data in units of packets can be selected in units of output areas, and addition and deletion of related code strings can be easily performed, output image quality management can be easily performed.

  Also, by making the code data such that the image reproduced by the reconstructed code data is equal to or greater than the distortion amount standard for each image attribute of the specific region in all image attributes, for example, reproduction of important character information Image quality can be improved.

  Next, when truncation is performed, if a different amount of truncation is performed for each code string corresponding to the image area (for example, for each code block in one subband) as described above, There is a case where distortion occurs (for example, between code blocks) and this appears as a distortion error.

  In such a case, the code amount may be controlled for each image region, or the code amount may be controlled uniformly for the entire image region. Of course, it may be performed in units of subbands by taking correspondences between code strings and subbands.

In the present invention, arithmetic code data to be truncated is determined based on the calculated importance calculation result of arithmetic code data as follows.
First, the importance and code amount in units of code strings (packets) constituting the code data are calculated, the code strings are arranged in the order of importance, and then the code quantity in ascending order of importance (not important). Are sequentially added, and the code string is deleted until the total code amount to be reduced is reached, and the code data is reconstructed.

  The code data deletion unit is arithmetic code data, but the unit may be another delimiter unit. In the case of code amount control in JPEG2000 encoding, deletion may be performed in units of code blocks or packets. It is only necessary that the importance is calculated for each unit.

  In the embodiment of JPEG2000 encoding according to the present invention, code data in code block units or packet units is composed of a set of arithmetic code data. Therefore, the importance of arithmetic code data units is increased by the method described above. If it is discriminated, the importance can be easily calculated by counting the results in units of code blocks or packets. Therefore, the code amount control information unit may be a JPEG2000 standard code block unit or a packet unit.

  Similarly, when code amount control is performed in units of partial image areas as described above, in JPEG2000 encoding, the target value for code amount control is JPEG2000 standard precinct units or tile units. The amount may be controlled.

  As described above, in the encoding method of the JPEG2000 (ISO / IEC 15444-1) standard, not only encoding is performed independently for each rectangular area (tile unit) of an image, but also a partial area unit (precinct) in each tile. Hierarchical code data is formed in (unit). Therefore, by providing a mechanism for managing the amount of code data (amount of truncation) in units of precincts, high-definition code amount control becomes possible.

<Distortion calculation method>

According to the present invention, priority is set for each code string of code data according to the distortion amount, and the code data in which the distortion amount of the code data corresponding to the specific image region is equal to or greater than the distortion amount reference is obtained for each image region. This is characterized in that no distortion exceeding the distortion amount standard is generated.
On the other hand, in JPEG2000 encoding, the amount of distortion is used for truncation in the encoding process. Therefore, in the present invention, the amount of distortion of the decoded reproduction image of the code data is calculated using the distortion amount calculated in the code data generation process. To do.

(1) JPEG2000 code string unit importance calculation

In the encoding process in the encoding based on the specification of the JPEG2000 standard (ISO / IEC 15444-1), the code forming process is performed in several code processing units (processing units such as subbands and code blocks). Truncation processing (code amount control) is performed in units of processing. At the time of code processing, importance is calculated for each code processing unit, and the code amount is controlled.
In the present invention, the degree of importance for each code string is calculated in consideration of the degree of feeling deterioration with respect to the distortion amount for each image attribute.

  By the way, as the coding processing unit of the above-mentioned specification of the JPEG2000 standard, there are a coding processing unit (subband or the like) meaning frequency information of image data and a coding processing unit (code block, precinct or the like) meaning an image area. .

Also, in the code processing of the JPEG2000 standard, arithmetic code data in units of code blocks created in the process of code formation using VW information taking into consideration visual characteristics based on the visual weighting method at the time of code processing The code amount is controlled.
Hereinafter, the encoding process of the JPEG2000 (ISO / IEC 15444-1) standard will be described sequentially, but more detailed description of the encoding of the JPEG2000 standard is described in Japanese Patent Application Laid-Open No. 2003-169333. .

(1-1) JPEG2000 encoding process

  JPEG2000 code processing does not require DC level shift and color conversion for each tile, wavelet conversion for each tile, quantization (normalization) for each subband, and bit-plane encoding for each code block In this configuration, a necessary code is discarded, a necessary code is collected to generate a packet, and the packets are arranged to form a code string. The decoding process is performed in the reverse procedure.

  FIG. 3 shows the relationship between image data to be encoded and tiles, subbands, precincts, and code blocks that are code forming units. A tile is a unit of image data obtained by dividing an image into rectangles. When the number of divisions is one, one image corresponds to a tile.

  Code data related to the precinct is generated in the encoding process, and corresponds to a partial area of an image (tile) as described later. The image area corresponding to the precinct corresponds to the image area corresponding to the code block constituting the precinct. Since the size of the code block is specified in advance and the way of separation is determined, the image area corresponding to the precinct can be obtained based on the image area corresponding to the code block.

  In the present invention, the attribute information for each image area of the original image is used after encoding. At this stage, the image attribute is determined for each partial image area corresponding to the code block. The image attribute for each code string (packet) of the formed code data is determined on the basis of the corresponding relationship.

In the JPEG2000 (ISO / IEC 15444-1) standard encoding method, attention is paid to the fact that actual images and code data are associated with each other in units of regions.
At this time, there is no problem if one image attribute is defined over the entire partial image area corresponding to the code block, but when a plurality of image attributes are included in the image area corresponding to one code block. The image attribute may be determined based on a predetermined rule.

  For example, in the method of defining image attributes in units of partial image areas corresponding to code blocks, an image is divided into units of partial image areas corresponding to code blocks after image input, and image attributes are defined in units of partial image areas. Here, the method of determining the image attribute is not limited to this. In addition to this, when there is already image data indicating the characteristics of an image, there is also a method in which an attributed image is divided into image regions and image attributes are determined in units of partial image regions.

Each tile is regarded as one piece of image data for each independent tile, and is subjected to code processing below the wavelet transform, so that all code data constituting the tile is an access unit in tile units.
As a result of converting the tiles from RGB to YUV or YCbCr by the color space conversion unit and performing wavelet conversion for each color component by the two-dimensional Wavelet conversion unit, subbands are generated and quantized in units of subbands.

  The precinct described above is a subband divided into rectangles (of a size that can be specified by the user) (collected for three subbands HL, LH, and HH, and one precinct is grouped into three. However, the precinct obtained by dividing the LL subband is a single unit), which represents a position in the tile (image).

  This precinct is generated as a result of wavelet transform of tile image data, and consists of code data of a certain area in the tile (image). Code data corresponding to a precinct access unit is code data of a precinct area in a tile (image), and a code block is obtained by further dividing the precinct into rectangles (of a size that can be designated by the user).

  In the following, the flow of generation of code data in units of code blocks and precincts in the course of JPEG2000 encoding processing will be described in order.

  First, the input image is subjected to wavelet transform in units of rectangular areas (tiles). The wavelet transform is applied to an input image (of a length necessary for filtering) in a specific area within the tile. This is realized by filtering (usually by a filter bank composed of a low-pass filter and a high-pass filter).

  The wavelet transform coefficients are sequentially formed corresponding to the input image in a specific area in the tile. That is, wavelet transform coefficients are formed corresponding to the image data of the specific image area in the tile. In this way, the image data is divided into tile units and encoded in tile units. However, as described below, in the encoding of the JPEG2000 specification, a partial rectangular image area of a smaller size in a tile. Wavelet transform coefficients are formed in units, and code data is formed in corresponding region units.

  As described above, the wavelet transform usually takes a method of repeatedly transforming a low frequency component (LL), and the wavelet transform coefficient is further subjected to wavelet transform. At this time, the wavelet transform is performed as data arranged at the position of the original image area in the tile used when forming the wavelet transform coefficients.

  This is because the wavelet transform is usually applied to data (collection) having a large correlation between image positions (regions) close to each other in order to increase the compression rate. For this reason, the wavelet transform coefficient for each subband maintains a correspondence with the position of the original image area in the tile.

Next, quantization processing is performed on the wavelet transform coefficients in units of subbands, and the obtained quantization coefficients also maintain a correspondence relationship with the position of the original image area in the tile.
Quantization performs lossy compression on wavelet transform coefficients. As a JPEG 2000 quantization means, scalar quantization is used in which a wavelet transform coefficient is divided by a quantization step size. Here, according to the JPEG2000 standard, when performing the above-described irreversible compression, if an irreversible 9 × 7 wavelet transform filter is used, it is determined that scalar quantization is automatically used together.

  On the other hand, when a reversible 5 × 3 wavelet transform filter is used, the code amount control is performed by truncating the coding pass as described later without performing quantization. JPEG2000 quantization processing is actually a case where an irreversible 9 × 7 wavelet transform filter is used.

  Subsequently, bit-plane coding is performed on the quantized coefficients. In the encoding standard in JPEG2000, entropy encoding processing called EBCOT (Embedded Coding with Optimized Truncation) is performed. EBCOT is a means for performing coding while measuring the statistical amount of coefficients in each block of a predetermined size.

  The quantized coefficients are entropy-coded into a block unit of a predetermined size called a code block (code-block). The code block is encoded independently for each bit plane from the MSB to the LSB of the quantization coefficient.

  A code block (code-block) is a collection of quantized coefficients having a correspondence relationship with the position of the original image area in the tile, and corresponds to a specific position of the original image area in the tile. It is a collection of quantized coefficients. The vertical and horizontal sizes of the code block are powers of 2 from 4 to 256, and commonly used sizes include 32 × 32, 64 × 64, 128 × 128, and the like. Code blocks are encoded in three types of encoding passes: (i) Significance Pass, (ii) Refinement Pass, and (iii) Cleanup Pass, in order from the bit plane on the MSB side. That is, each bit plane is encoded in the order of Cleanup Pass, Significance Pass, Refinement Pass, and Cleanup Pass from the MSB side to the LSB side of the bit plane. Each of the three arithmetic encodings performs the following processing.

(I) Significance Pass:
In the Significance Pass that encodes a certain bit plane, the bit plane value of a non-significant coefficient in which at least one coefficient in the vicinity of 8 is a significant is arithmetically encoded. When the value of the encoded bit plane is 1, arithmetic coding is continued by determining whether the code is + or-.

(Ii) Refinement Pass:
In the Refinement Pass that encodes the bit plane, the value of the bit plane of the significant coefficient that is not encoded by the Significance Pass that encodes the bit plane is arithmetically encoded.

(Iii) Cleanup Pass:
In the Cleanup Pass that encodes the bit plane, the bit plane value of the non-significant coefficient that is not encoded by the Significance Pass that encodes the bit plane is arithmetically encoded. When the value of the encoded bit plane is 1, whether the code is + or-is continuously arithmetic encoded.

  Bit-plane coding is performed independently for each block (code block unit) corresponding to a specific position of the original image area in the tile, and the arithmetic coding statistic measurement is performed on the coded block ( In the above-described example, the processing is performed while being closed in the code block).

  As described above, after the arithmetic code is generated in each coding pass, the code amount control is performed so as to approach the target bit rate or compression rate while counting the code amount of the arithmetic code as necessary. This control of the code amount can be realized by truncating (truncating) a part or all of the arithmetic code data (the arithmetic code data generated in each encoding pass) for each code block.

  The code amount control method of the present invention will be described in detail later, but the bit plane encoding unit creates a code before code discard and performs truncation (by the post quantization unit) while adjusting the code amount. The packet sequence is sequentially generated to form a code in the JPEG2000 code format.

  After the code amount control is completed, the code data is generated by editing the arithmetic code data in units of code blocks. That is, the subband coefficients after quantization are bit-plane encoded in units of code blocks (one bit plane may be decomposed into three sub-bit planes and encoded).

  For example, additional information is generated as a header for arithmetic code data in a code block. Normally, header information constituting code data is generated, while on the other hand, arithmetic code data in units of code blocks is edited to generate a packet, and corresponding packet data is added to the header information to generate code data.

  The packet header usually includes information specifying the code block included in the precinct, the number of 0 bit planes not encoded, the number of encoded coding passes, or zero length packets (all code block information is 0). ) Have information about

  In code data compliant with the JPEG2000 specification, a packet is configured by collecting code blocks corresponding to a rectangular area (position) in a tile called a precinct. That is, a packet is obtained by collecting a part of the code from all the code blocks included in the precinct (for example, collecting the codes of the bit planes from the MSB of all the code blocks to the third one). . However, the content of the packet may be “empty” in terms of code.

  When packets of all precincts (= all code blocks = all subbands) are collected, a part of the code of the entire image (for example, the code of the bit plane from the MSB to the third frame of the wavelet coefficients of the entire image) This is called a layer. The layer is a part of the code of the bit plane of the entire image, and when all the layers are collected, the code of all the bit planes of the entire image is obtained.

  FIG. 4 shows layers when the number of wavelet transform layers (decomposition level) = 2 and precinct size = subband size, and FIG. 5 shows an example of a packet included therein. In these cases, the precinct size is equal to the subband size, and the code block having the same size as the precinct size shown in FIG. 3 is used. Therefore, the subband at the decomposition level 2 is divided into four code blocks. The sub-band of decomposition level 1 is divided into nine code blocks. Since the packet is based on the precinct, when precinct = subband, the packet spans the HL to HH subbands. In FIG. 5, some packets are surrounded by thick lines. These packets are “a collection of code blocks extracted and collected”.

  Examples of JPEG2000 packet configurations are shown in FIGS. The packet is the minimum unit of code data of the JPEG2000 (ISO / IEC 15444-1) standard, and is composed of code data in precinct units.

  In this way, code data formed by JPEG2000 encoding is formed in tile units, further, code data in tile units is formed in packet units, and packets are formed by a collection of code strings in precinct units. Has been. Therefore, when accessing a code string formed by JPEG2000 encoding in units of precincts, access can be made by analyzing the header of the code string and extracting packets corresponding to the designated precinct.

  By the way, in the encoding of the JPEG2000 (ISO / IEC 15444-1) standard, packets are further collected into several structural units to form code data (encoded code stream). The sequence of packets is called a progression order, In the JPEG2000 (ISO / IEC 15444-1) standard, code data is configured in several progression orders.

In other words, JPEG2000 (ISO / IEC 15444-1) standard packets are sequenced in a progressive order, and are arranged by precinct, resolution level, component (color component), and image quality level (layer), respectively.
Here, although the component has not been described in detail so far, the component refers to a color component, and corresponds to encoding performed for each color component in tile units of the image. When accessing code data of a certain tile or precinct, the corresponding code data is accessed in all the components of the corresponding tile area.

  In addition, the resolution level refers to the subband hierarchy (decomposition level), and as described above, arithmetic code data in units of code blocks is generated for the wavelet coefficients obtained for each subband. Code data can be distinguished for each subband hierarchy (decomposition level). Similarly, when accessing code data of a certain tile or precinct, the corresponding code data is accessed at all resolution levels of the corresponding tile area.

  Also, the image quality level (layer) corresponds to the image quality level, for example, 3LL code block unit data is configured as the lowest level layer, and 3LL to 3HH is added to the 3LL code block data. Layer data is configured with a layer, data in units of code blocks to which 2HL to 2HH is added as the next layer, and data in units of code blocks to which 1HL to 1HH is added to 2HL to 2HH as the highest layer. Similarly, when accessing code data of a certain tile or precinct, the corresponding code data is accessed at all layer levels of the corresponding tile area.

  In the JPEG2000 standard, by changing the priority order of the four image elements of image quality (layer (L)), resolution (R), component (C), and position (precinct (P)), the following five patterns are displayed. Progression is defined.

LRCP progression: Since the decoding is performed in the order of precinct, component, resolution level, and layer, the image quality of the entire image is improved each time the layer index is advanced, and the image quality progression can be realized. Also called layer progression.
RLCP progression: Since the decoding is performed in the order of precinct, component, layer, and resolution level, resolution progression can be realized.
RPCL progression: Since decoding is performed in the order of layer, component, precinct, and resolution level, it is a progression of resolution level as in RLCP, but the priority of a specific position can be increased.
PCRL progression: Since decoding is performed in the order of layer, resolution level, component, and precinct, decoding of a specific part is prioritized, and progression of spatial position can be realized.
CPRL Progression: Since decoding is performed in the order of layer, resolution level, precinct, and component, for example, it is possible to realize component progression that first reproduces a gray image when progressively decoding a color image.

  As described above, in the coding according to the JPEG2000 specification, the code order (progressive method) of the code data to be formed can be selected. As the progressive method, resolution (resolution level), precinct (position), color Five progressive orders based on combinations of components and layers are defined.

  In general, the multi-layer progressive has a progressive order of Layer-resolution level-component-position (LRCP), and encodes a bit plane from the highest level. When an image is displayed in this multi-layer progressive manner, it seems to the user that the number of colors gradually increases from a small number of colors. When this is applied to a document in which characters, photos, etc. are mixed, the photo cannot be recognized unless it is restored almost completely because the halftone is not visible. On the other hand, since characters have strong edges, they can be recognized before they are completely restored.

  On the other hand, multi-resolution progressive has a progressive level of resolution level-Layer-component-position (RLCP), and uses the multi-resolution property of wavelet transform to encode from a small resolution. Is compressed with the fineness and roughness of the image. In decoding, when an image is displayed using multi-resolution progressive resolution that gradually increases from a low resolution, it appears to the user that the focus is gradually in focus from the out-of-focus state. If this is applied to a document in which characters and photographs are mixed, the photograph can recognize the rough contents before being completely restored, and the characters cannot be recognized unless the characters are almost completely restored.

In encoding according to the JPEG2000 specification, the code amount may be controlled by forming a code string based on the code order (progressive method) selected in this way and deleting a part of the hierarchy after encoding. it can. In code amount control after encoding, since each layer of code data is generally composed of a plurality of code strings, fine code amount control cannot be performed.
As described in the present invention, by calculating the importance of the code data to be formed in units of code strings (because the priority order in units of code strings (packets) is clear), It becomes possible to first delete an unimportant code string in units of code strings in the code data in the hierarchy.

  That is, as one embodiment of the present invention, after a code order (progressive method) is selected and code data having a hierarchical structure is formed, priorities are assigned to the hierarchies based on the importance of the code string. Code amount control for sequentially deleting from a low code string is also included.

  The JPEG2000 standard code data formed in this way generates encoded data that is divided into tile units by precinct, resolution level, image quality level (layer), and color component. Regardless of the order of progression, when accessing code data in a certain tile unit, the corresponding code data is accessed for all precincts, resolution levels, image quality levels (layers), and components of the corresponding tile area.

  On the other hand, when accessing the code data of the image area corresponding to the precinct unit, the corresponding code data is accessed with all resolution levels, image quality levels (layers), and components corresponding to the precinct unit.

  In other words, code data complying with the JPEG2000 specification is accessed in units of tiles (image regions) obtained by dividing the entire image into units of rectangular regions (when accessing code data included in tiles), and in units of regions within the tiles. Two types of code data can be accessed, that is, code data (in a corresponding image area) is accessed in a certain precinct unit. The specification is such that code data having a delimiter unit of access units that can easily access image region units having two types of sizes is formed.

  Therefore, it is possible to analyze the header of code data generated in accordance with the JPEG2000 specification, and to truncate (code string control) or edit the code data in tile (image area) units or precinct units.

(1-2) Distortion amount calculation means 1:

Next, code amount control in the encoding process in JPEG2000 will be described.
In the encoding conforming to the JPEG2000 specification, a visual weighting (VW) method is used to realize a code amount control method.
The VW method is a technique that skillfully uses the human visual system, and models the fluctuation sensitivity of the human visual system with respect to the spatial frequency of the image, and systematizes this as a contrast sensitivity function (CSF). It is a thing.

  The CSF weight is actually determined by the visual frequency of the transform coefficient of the image, increasing the bit allocation for the coefficient within the visible wavelength band and increasing the bit allocation for the coefficient within the non-visible wavelength band. It is a visual weighting that reduces, and enhances the subjective perception of the image by enhancing features that are more perceived by the human eye. A CSF weight is separately prepared for each subband after wavelet transform.

  The CSF weight is designed on the encoding side and depends on the conditions under which the decoded image is viewed. Specifically, as will be described below, a VW coefficient value is set for each subband, and the arithmetic code data of the coding pass of the code block that is a reduction evaluation target based on the set VW coefficient value The amount of distortion reduction is calculated, and the amount of code is controlled so as to preferentially reduce the arithmetic code data having a small amount of distortion reduction. Thus, the importance of each code block can be calculated based on the distortion reduction amount of the arithmetic code data constituting the code block.

  In the following description, a code amount control process by truncating a part of code data by the CSF weight will be described. However, there is a case where the quantization step size is adjusted by the reciprocal number of the CSF and the code amount is controlled by quantization. .

As illustrated in FIG. 9, the coding pass code amount calculation unit 300 and the coding pass distortion reduction amount calculation unit 310 receive the arithmetic code D340 after arithmetic coding.
The coding pass code amount calculation unit 300 obtains the code amount R actually generated in the coding pass generated in the code block for all the coding passes of all the code blocks, and all the code blocks The information D300 indicating the code amount R of the encoding pass is supplied to the RD characteristic calculator 320.

  The encoding pass distortion reduction amount calculation unit 310 calculates a distortion reduction amount D indicating how much distortion is reduced when the encoding pass is included. The distortion amount is an error with respect to the original image data of the image data decoded in a state where the target coding pass is truncated. The error is calculated using MSE (Mean Squared Error). As described above, in JPEG2000, in order to obtain the amount of distortion when truncating to each bit plane, decoding is performed in a state where each bit plane is truncated, and an error is examined by MSE.

  For example, first determine the amount of distortion when truncating one bit plane from the lower side in each code block, and then determine the amount of distortion when truncating two bit planes from the lower side. The amount of distortion when truncating the bit plane is obtained. For this reason, there is a problem that the processing time for obtaining the distortion amount becomes very long, and the distortion amount may be obtained by a simple method as described later.

  At this time, in the encoding process compliant with the JPEG2000 specification, the distortion reduction amount D is calculated in consideration of the VW coefficient value D330. In other words, the distortion reduction amount D is obtained by multiplying the distortion reduction amount of each coding pass by the VW coefficient value D330 set for each subband. This is equivalent to preferential weighting for each subband, and the coding pass existing in the low-frequency subband is less likely to be discarded during code amount control, and conversely the high-frequency subband. Existing encoding passes are more likely to be truncated. The reduction amount D is obtained for all the coding passes of all the code blocks, and information D310 indicating the distortion reduction amount D of the coding pass of all the code blocks is supplied to the RD characteristic calculation unit 320.

  The RD characteristic calculation unit 320 calculates the RD characteristic of each coding pass, usually the slope of the RD curve. Then, the RD characteristic calculation unit 320 stores and holds the RD characteristic value and the information D320 of each coding pass obtained as a result for all the coding passes, and the RD characteristic value and the information of each coding pass. D320 is supplied to the encoding pass selection unit 330.

Based on the RD characteristic value supplied from the RD characteristic calculation unit 320 and the information D320 of each coding pass, the coding pass selection unit 330 sets each coding pass so as to be closest to the target code amount of the entire image. Select whether to include or truncate. At this time, the encoding pass selection unit 330 uses RD characteristics as shown in FIG. In FIG. 10, black circle points indicate encoding passes that are candidates for truncation, and white circle points indicate encoding passes that are not subject to truncation.
Therefore, the coding pass selection unit 330 performs an operation of selecting these white circle coding passes so as to be closest to the target code amount.
Through the above operation, the arithmetic code D350, which is information of the finally selected coding pass, becomes a code formation target.

  In addition, when the target code amount is reached, or just before reaching the target code amount, the coding pass can be discarded, or the code amount can be controlled by terminating the coding pass without selecting any more.

  In the encoding conforming to the JPEG2000 specification, the code amount control by the VW method is executed using the VW coefficient value D330 set for each subband. The actual value of the VW coefficient value D330 can be determined for each subband of each division level as shown in FIG. 11, for example. In FIG. 11, coefficient values are determined separately for luminance (Y) and color differences (Cb, Cr), and the coefficient value increases as the division level increases, that is, the lower frequency component. I understand. Here, a large coefficient value means that the encoding pass is not easily truncated during code amount control.

  In FIG. 11, the coefficient value increases as all of Y, Cb, and Cr are lower (the higher the division level), the fact that the energy of the image is concentrated in the lower frequency. Because it is. Here, the coefficient value of a specific subband may be set to a large value or a small value using the characteristics of the input image. For example, when the input image is an interlaced image, the energy of the LH component tends to increase. Therefore, it is effective to increase the weight value of the coefficient corresponding to the LH component as shown in FIG. As a result, the image of the interlace component is retained, and there is an effect that leads to high image quality.

  11 and 12, the weight coefficient value of Cb or Cr is set to a smaller value than Y. This is because the human visual characteristic with respect to the color difference component is utilized rather than the luminance component.

  Code amount control is not limited to the JPEG2000 specification shown above, but when a bit plane is truncated based on the number of most significant bits in each bit plane of image information, as in Japanese Patent Laid-Open No. 2003-304405. This method can also be realized by estimating the amount of image distortion and truncating the image.

In the importance setting in the code string unit of the code data of the present invention, a standard for each image attribute is used. As described above, since the image area information can be identified for each code block, when calculating the distortion reduction amount D, the image area information is set in consideration of the image area information of the target code block. Whereas the VW coefficient value is set for each subband, the image attribute information for each image area is set for each code block.
In this case, the distortion reduction amount of each coding pass is multiplied by the VW coefficient value D330 set for each subband and the value set for each image attribute for each image area for each code block. A reduction amount D is assumed.

  From the above, the present invention provides a subband generating means for generating a subband by applying a low-pass filter and a high-pass filter to an input image in a vertical direction and a horizontal direction, and a low-frequency component. Filtering means for hierarchically filtering the subbands, code block generation means for dividing the subband generated by the filtering means to generate a code block of a predetermined size, and for each code block Bit plane generating means for generating a bit plane from the most significant bit to the least significant bit, coding path generating means for generating a coding pass for each bit plane, and arithmetic for performing arithmetic coding in the coding pass Encoding means and arithmetic code distortion amount calculating means for calculating the distortion amount of the arithmetic code, and calculating the calculated distortion amount And so that the strain amount of the decoded reproduced image of the code data in the bright.

  (1-3) Strain amount calculation means 2

  As described above, in JPEG 2000, quantized wavelet coefficient data is divided into bit planes and encoded, so that image information can be compressed (code amount control) by truncating the bit planes. For example, image information compression (code amount control) is performed by truncating the bit plane from the lower side (truncation). In the code amount control, when a certain compression rate exists as a target value, data is truncated until the target value is reached. Naturally, when the data is truncated, the image quality deteriorates. For this reason, when data is cut off, it is necessary to detect how much data is cut off and how much the image quality deteriorates. The degree of the deterioration is the distortion amount.

  The detection of the distortion amount is not limited to the one based on the JPEG2000 specification, and an image when truncating a bit plane based on the number of most significant bits in each bit plane of image information as disclosed in Japanese Patent Laid-Open No. 2003-304405. This method can also be realized by estimating the amount of distortion and truncating. Based on the same idea, the distortion reduction amount of each coding pass may be calculated based on the upper significant bits (values) of each coding pass.

  That is, as a method for detecting distortion, in an image processing method for compressing image information by truncation of a bit plane of image information, such as JPEG 2000, the distortion amount of an image when a bit plane (or layer) is truncated is used. , The amount of change in the number of MSBs (most significant bits of data) when truncation is performed.

  This means that when truncating a bit plane (layer) in which many MSB components of wavelet coefficients are distributed, the distribution status of code block data relating to that bit plane (layer) will change significantly. This is because it is considered that there is a correlation between the change in the number of MSBs and the change in the distortion amount when truncation of a plane (layer) is performed.

  From the above, according to the present invention, the amount of distortion is detected by encoding in which the amount of code is reduced by truncation in units of bit planes for each image area in the process of generating code data. Here, the distortion amount is the number of most significant bits in each bit plane for each image area.

(2) JPEG2000 code string unit importance setting

  In the coding process of the JPEG2000 (ISO / IEC 15444-1) standard, the code data is formed in units of subbands corresponding to the frequency characteristics in the coding process. Thereafter, the code data is complicated in the process of the coding process. Therefore, after the encoding process, the code data cannot be distinguished in units of subbands, and the importance level is set in units of code strings considering the visual characteristics of such units. Can not.

  Therefore, in the configuration of the present invention, as a method of setting the importance in units of code strings constituting the code data after encoding, the distortion amount is calculated in units of code blocks as described above during the encoding process. Then, after encoding, importance is calculated for each code string unit in code formation based on the distortion amount using the information. In JPEG2000 encoding, since a code string (packet) in code formation is formed by a set of code block units, the importance of the code string (packet) in code formation is the code forming the code string. Calculated as the sum of weights in block units.

  Specifically, in the process after encoding, the distortion reduction amount D and the code amount R value for the arithmetic code data of the encoding pass of the code block calculated in the encoding process are stored. use. At this time, the calculation of the distortion reduction amount may be a calculated value considering the VPEG coefficient value D330 of the JPEG2000 (ISO / IEC 154444-1) standard, or may be a value considering the image attribute for each image region.

  The setting of the importance level in units of code strings in the present invention will be described with reference to FIG. The present embodiment is characterized in that it is provided with means for calculating a distortion amount in units of code blocks, and has a function of setting importance in units of code strings (packets) at the time of editing or decoding of encoded data after encoding. .

The encoding processing unit 410 includes an arithmetic code calculation unit 411 in units of code blocks for each subband that is temporarily generated in the course of encoding, and the arithmetic code storage unit 412 in units of code blocks for each subband includes the arithmetic code storage unit 412. The code is stored.
The code block unit distortion amount calculation unit 403 calculates the distortion amount of the arithmetic code in code block units.

  The code string unit importance calculation unit 400 uses the data stored in the image attribute information storage unit 401 and the VW coefficient storage unit 402 in the code block corresponding image region unit, and the arithmetic code of the code block unit in each subband. A code block unit distortion amount calculation unit 403 that calculates a distortion amount (degradation degree of image quality when the arithmetic code data is reduced) includes a code string (packet) unit distortion amount calculation unit 404. ) The distortion amount of the unit is calculated, and the importance of the arithmetic code of the code string (packet) unit is calculated by the code string unit importance calculation unit 405 using the distortion amount of the code string (packet) unit.

In the configuration of FIG. 13, the code amount calculation unit 406 is included in the code string unit importance calculation unit 400, but may be included in the encoding processing unit 410.
As shown in FIG. 13, in the present invention, the degree of importance is set in units of code strings using the distortion amount in units of code blocks.
In order to set the importance level in units of code strings (packets) in JPEG2000 encoding, the following procedure is used.

(i) Input image data, wavelet transform the image data, and calculate wavelet coefficients in subband units.
(ii) The following processes (iii) to (vi) are repeated for each subband wavelet coefficient unit.
(iii) Arithmetic code data is calculated for each code block, and the following processes (iv) to (v) are repeated.
(iv) A distortion amount is calculated for each piece of arithmetic code data included in the code block, and the distortion amounts of the code code block are calculated by adding the distortion amounts.
(v) When the processing of all arithmetic code data is completed, the next code block unit is processed.
(vi) When the processing for all code blocks is completed, the next subband unit is processed.
(vii) When the processing in units of all subbands is completed, code data in code block units are combined to form a code string to form code data (packet units).
(viii) The amount of distortion in units of code blocks constituting the code string is added up to calculate the amount of distortion in units of precinct.
(ix) The calculated distortion amount in units of precinct is used as the distortion amount in units of code strings (packets).
(x) The degree of importance for each code string (packet) is calculated by comparing the calculated distortion amount for each code string with a distortion amount reference value.

  That is, in the code data formation process of JPEG2000, when generating a packet composed of code data in units of precinct, the amount of distortion of arithmetic code data in units of code blocks constituting the precinct is totaled to calculate the amount of distortion in units of precinct. Calculated.

In any case, in the code data editing process (parser) after forming the code string, the distortion amount information in the precinct unit or the distortion amount information or importance information in the code string (packet) unit is left as a record when the code data is formed. In addition, the information can be used not only for the subsequent decoding process, the image process after decoding, or the access of the code data after decoding.
Here, as a method of holding the distortion amount information or the importance level information, it may be described in the header of the code data, or may be held separately as a table.

  Further, in the image processing apparatus according to the embodiment of the present invention, as shown in FIG. 2, the code data is constituted by the designated area image attribute information, the designated area distortion amount, and the distortion amount standard for each image attribute. The importance (priority order) is calculated for each code string.

  Similarly, it was previously shown that image attributes are defined in units of partial image areas, and importance is set for controlling the code amount for each image area. However, another table is also provided for image attributes in units of such partial image areas. It may be held in the form of Since these pieces of information have correspondence between the original image area and the code data, the code data processing for each image area can be easily performed when using these pieces of information.

(3) Importance setting using image attribute information of JPEG2000

FIG. 14 is a diagram for explaining importance level setting in units of code strings using a distortion amount criterion based on image attributes in the image processing apparatus according to the embodiment of the present invention. It shows how the importance of the code data is set for each image attribute of the partial area constituting the image data.
As shown in FIG. 14, in the present invention, a distortion amount and attribute information for each code string are calculated based on image attribute information for each image area calculated in the encoding process and distortion amount reference information for each image attribute. Later, importance level setting information for each code string is calculated, and code data editing processing is performed.

As described above, the distortion amount in units of code strings (the distortion amount of image quality that affects the reproduction image quality when the code string is deleted) and the attribute information are calculated in the encoding process as described above. The importance of the code string unit is less important when the distortion amount of the code string is compared with the distortion amount standard for each image attribute and the distortion amount is determined to be smaller than the reference value of the image attribute. Otherwise, it is set to be large.
In the image attribute information calculation in the coding process of the present invention, an image is divided into partial image area units corresponding to code blocks, image attributes are calculated in code block units, and image attributes are calculated in code string (packet) units. .

  As described above, the importance of the code string constituting the code data reflects the difference in sensitivity to the distortion amount with respect to the image attribute, in particular, by changing the setting criteria for the importance of the code data depending on the image attribute. Can do.

  In the method of calculating the frequency characteristics such as JPEG2000 in the encoding process, the frequency characteristics calculated in the encoding process are used instead of determining the image attributes for each partial image area by the method described above. Thus, the image attribute may be calculated for each partial image area corresponding to the code block.

For example, the encoding processing unit performs wavelet transform on image data, calculates wavelet coefficients in subband units, uses LL component data to calculate image attributes in units of partial image regions corresponding to code blocks, The image attribute may be calculated for each column (packet).
In the above description, when an image attribute is defined for each partial image area, it may be erroneously identified. In accordance with such a case, the setting range of the important area may be set to be wider than the calculated area.

  Such attribute information of the image is extracted by an image area separation technique such as a scanner. At this time, as described above, the image attribute is discriminated in units of partial image areas, and the correspondence with the code data constituting the code string is taken.

The present invention is not limited only to the above-described embodiments. Each function constituting the image processing apparatus and the image processing system of the above-described embodiment is programmed and written in a recording medium in advance, and these programs recorded in the recording medium are stored in a memory or storage provided in the computer. Needless to say, the object of the present invention is achieved by storing the program in the apparatus and executing the program. In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the program and the recording medium recording the program also constitute the present invention.
In addition, the program includes a case where the functions of the above-described embodiment are realized by processing in cooperation with an operating system or another application program based on an instruction of the program.

Note that the program for realizing the functions of the above-described embodiments includes a disk system (for example, a magnetic disk, an optical disk, etc.), a card system (for example, a memory card, an optical card, etc.), and a semiconductor memory system (for example, a ROM, a nonvolatile memory). Etc.) and a recording medium of any form such as a tape system (for example, magnetic tape, cassette tape, etc.). Alternatively, the program stored in the storage device may be directly supplied from the server computer via the network. In this case, the storage device of this server computer is also included in the recording medium of the present invention.
As described above, by programming and distributing the functions of the above-described embodiment, the cost can be reduced, and the portability and versatility can be improved.

It is a block diagram which shows the structure of the importance level setting in the image processing apparatus which concerns on embodiment of this invention. 1 is a block diagram illustrating a configuration of an image processing system according to an embodiment of the present invention. It is a figure which shows the relationship between the image data which are encoding objects, and the tile, subband, precinct, and code block which are code formation units. It is an example of a layer structure in JPEG2000. It is a structural example of a packet in JPEG2000. It is a structural example (the 1) of the packet data in JPEG2000. It is a structural example (the 2) of the packet data in JPEG2000. It is a structural example (the 3) of the packet data in JPEG2000. It is a figure which shows the flow of the code amount control processing in JPEG2000. It is an example of an RD characteristic (relationship between distortion amount and code amount) table in a code amount control process of JPEG2000. It is an example of the weight coefficient table of the VW method in the code amount control process of JPEG2000. It is an example of another weighting coefficient table of the VW method in the code amount control processing of JPEG2000. It is a block diagram which shows the other structure of the importance level setting in the image processing apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the importance level setting of the code sequence unit using the distortion amount reference | standard based on an image attribute in the image processing apparatus which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Input processing part, 20 ... Coding process part, 30 ... Code sequence unit importance calculation part, 31 ... Code string unit attribute information extraction part, 32 ... Code string unit distortion amount extraction part, 33 ... Code string unit importance Setting unit 34... Image attribute distortion amount reference storage unit 35. Image attribute information storage unit 40. Code data editing unit 50. Decoding processing unit 60. Output processing unit 70 70 Image data storage unit 1 80 ... Code data storage unit, 90 ... Image data storage unit 2, 100 ... Server, 110 ... Image area designation part, 120 ... Area range corresponding code data selection part, 130 ... Image attribute information extraction part in designated area, 135 ... Area unit Image attribute information storage unit, 140... Designated area distortion amount calculation unit, 145... Region unit distortion information storage unit, 150... Code string selection unit, 155 .. code data storage unit, 165. Stream generator, 00 ... client, 300 ... coding path code amount calculation unit, 310 ... reduction amount calculation unit, 320 ... RD characteristic calculation unit, 330 ... coding path selection unit, 400 ... code string unit importance calculation unit, 401 ... code block Corresponding image area unit image attribute information storage unit 402... VW coefficient storage unit 403... Code block unit distortion amount calculation unit 404. Code string unit distortion amount calculation unit 405. Code amount calculation unit, 410 ... encoding processing unit, 411 ... code block unit arithmetic code calculation unit for each subband, 412 ... code block unit arithmetic code storage unit for each subband, 420 ... code data editing processing unit, 421 ... code amount Control processing unit, 430 ... code amount storage unit, 440 ... code string unit importance storage unit.

Claims (21)

  A function of setting importance (priority order) in units of code strings (packets) constituting the code data based on the distortion amount of the decoded reproduction image of the code data, the image attribute information, and the distortion amount standard for each image attribute An image processing apparatus.   The code amount corresponding to the image partial area, the amount of distortion of the decoded reproduction image, the image attribute information, and the distortion amount standard for each image attribute, in units of code strings (packets) constituting the code data corresponding to the image partial area An image processing apparatus having a function of setting importance (priority order).   The image processing apparatus according to claim 1, wherein the code processing unit has a function of changing the order of the code strings constituting the code data and reconfiguring the code data according to the importance (priority order) of the code string (packet) unit. An image processing apparatus.   The image processing apparatus according to claim 1, wherein the image processing apparatus has a function of transmitting in order of code strings in units of code strings constituting code data according to the importance (priority order) of the code strings (packets). An image processing apparatus.   5. The image processing apparatus according to claim 4, wherein the code string is cut off in the middle according to the transmission status and transmitted in units of code strings constituting the code data according to the importance (priority order) of the code strings (packets). An image processing apparatus having a function.   6. The image processing device according to claim 4, wherein a part of the code string constituting the code data is replaced with a dummy code string having a short length to constitute code data to be decoded. Image processing device.   4. The image processing apparatus according to claim 3, further comprising means for comparing a distortion amount for each image area with a distortion amount reference, so that the code data corresponding to the area having the specific image attribute is greater than or equal to the distortion amount reference. An image processing apparatus comprising means for reconstructing code data.   The image processing apparatus according to claim 3, further comprising a unit that compares a distortion amount for each image attribute with a distortion amount reference, and an image reproduced by the reconstructed code data has a specific region in all image attributes. An image processing apparatus characterized by being code data that is equal to or greater than a distortion amount standard for each image attribute.   3. The image processing apparatus according to claim 2, wherein the partial area of the image is a preset attention (ROI) area.   10. The image processing apparatus according to claim 1, wherein the distortion amount of the decoded reproduction image of the code data according to claim 1 or 2 is a distortion amount calculated in a code data generation process. An image processing apparatus.   11. The image processing apparatus according to claim 10, wherein the code data generation process according to claim 1 or 2 applies a low-pass filter and a high-pass filter to an input image in a vertical direction and a horizontal direction to generate subbands. A subband generating means for generating, a filtering means for performing a hierarchical filtering process on the subbands of the low-frequency component, and a subband generated by the filtering means is divided into code blocks of a predetermined size. Code block generating means for generating, bit plane generating means for generating a bit plane from the most significant bit to the least significant bit for each code block, and an encoding path generating means for generating an encoding path for each bit plane. Arithmetic coding means for performing arithmetic coding in the coding pass, and arithmetic for calculating the distortion amount of the arithmetic code. An image processing comprising: a code distortion amount calculation unit, wherein the distortion amount of the decoded reproduction image of the code data according to claim 1 or 2 is a distortion amount calculated by the arithmetic code distortion amount calculation unit apparatus.   The image processing apparatus according to claim 10, wherein in the code data generation process according to claim 1 or 2, encoding is performed to reduce a code amount by truncation in units of bit planes for each image area. An image processing apparatus characterized in that the number of most significant bits in each bit plane is a distortion amount.   13. The image processing apparatus according to claim 1, further comprising means for designating an image area and means for extracting code data in a range corresponding to the image area.   14. The image processing apparatus according to claim 13, wherein means for designating an image partial area and means for reconstructing code data based on importance of a code string (packet) constituting code data corresponding to the image area An image processing apparatus comprising:   15. The image processing apparatus according to claim 1, wherein the code data is data encoded based on JPEG2000 (ISO / IEC 15444-1) standard.   The image processing apparatus according to claim 15, wherein the image partial area according to claim 2 is a precinct unit or a tile unit of JPEG2000 (ISO / IEC 15444-1) standard.   16. The image processing apparatus according to claim 15, wherein the code string unit described in claim 1 or 2 is a packet unit of JPEG2000 (ISO / IEC 15444-1) standard.   The importance (priority order) is set in units of code strings (packets) constituting the code data based on the distortion amount of the decoded reproduction image of the code data, the image attribute information, and the distortion amount standard for each image attribute. An image processing method.   The code amount corresponding to the image partial area, the amount of distortion of the decoded reproduction image, the image attribute information, and the distortion amount standard for each image attribute, in units of code strings (packets) constituting the code data corresponding to the image partial area An image processing method characterized by setting importance (priority order).   A program for causing a computer to realize the functions of the image processing apparatus according to any one of claims 1 to 17.   A computer-readable recording medium on which the program according to claim 20 is recorded.