JP6596837B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Patent Document 1 provides a video transmission terminal and a video transmission terminal capable of realizing the scalability of encoded data obtained by object video encoding without degrading the quality of an important part of a video. The priority area is set from the important part of the area extraction data extracted by the area extraction section for the subject of the digital video obtained from the video source, and the object video encoding section is encoded for the set priority area. Thus, it is disclosed that scalability is realized for encoded data, and even if the encoded data is reproduced, important portions can be displayed without degrading the quality.

  In Patent Document 2, it is an object to compress and encode image data with a small amount of computation and at high speed, and to obtain a compressed image that matches the target image quality while removing noise. In addition, it is disclosed that data is truncated so that a desired noise removal effect is obtained with respect to a code string that has been bit-shifted, and the data is truncated sequentially from the rightmost bit.

  In Patent Document 3, scalable video is transmitted according to priority to solve the problem of optimizing selection and transmission of a bit stream of scalable video, and finally improving the quality of an image reconstructed from the bit stream The priority is rearranged for the layered bitstream after the SVC encoding of each GOP of the scalable video, and the package and the Raptor encoding are performed for the SVC layered bitstream after the rearrangement. And transmitting the encoded scalable video bit stream to the scalable video receiver via the transmission channel.

JP 2001-066952 A JP 2007-104645 A JP 2011-147120 A

As a method for compressing an image, there is Progressive-JPEG or the like. In these techniques, a rough low-frequency image is first displayed over the entire image, and a fine high-frequency image is gradually displayed. Therefore, the first basic image does not include a high-frequency component.
Therefore, since the first image does not contain a high frequency component, an edge is lost.
For example, a phenomenon occurs in which the recognition rate of a character image decreases in the first image.
It is an object of the present invention to provide an image processing apparatus and an image processing program that improve the image quality of an image after the first decompression as compared with Progressive-JPEG.

The gist of the present invention for achieving the object lies in the inventions of the following items.
The invention according to claim 1 is a difference image generating means for generating a difference image between a quantization means for performing quantization while preserving an edge of an image and the image and the quantized image quantized by the quantization means. A first compressing unit that compresses the quantized image quantized by the quantizing unit, a second compressing unit that compresses the differential image generated by the differential image generating unit, and the differential image generating unit Control means for controlling the distance in the cluster formation by the quantization means for performing the limited color processing so that the processing result by the predetermined range falls within a predetermined range , the quantization means is a clustering as quantization class have rows color quantization process, the control unit may exceed the value distance is predetermined to the centroid of each cluster generated by the clustering by Do not include the, adds the centroid intermediate values range of the pixel, it is an image processing apparatus according to claim.

The invention according to claim 2 is a difference image generating means for generating a difference image between a quantization means for performing quantization while preserving an edge of an image and the image and a quantized image quantized by the quantization means. A first compressing unit that compresses the quantized image quantized by the quantizing unit, a second compressing unit that compresses the differential image generated by the differential image generating unit , and the differential image generating unit The control unit for controlling the distance in the cluster formation by the quantizing unit that performs the limited color processing and the code generated by the first compression unit are expanded so that the processing result by the step falls within a predetermined range. First decompression means, second decompression means for decompressing the code generated by the second compression means, and an image decompressed by the first decompression means to the second decompression means. Comprising a summing means for adding the decompressed image I, the quantization unit, as a quantization, have rows color quantization process by clustering, the control means, each cluster generated by the clustering The image processing apparatus is characterized in that the distance to the center of gravity of the pixel is not included in a cluster exceeding a predetermined value, and an intermediate value of the pixel value range is added to the center of gravity .

The invention according to claim 3, wherein the first compression unit performs lossless compression process, the second compression means, the image processing according to claim 1 or 2, characterized in that the lossy compression Device.

The invention of claim 4 further comprises second quantization means for quantizing a flat portion of the image, wherein the quantization means is a second quantized image quantized by the second quantization means. as target, claim 1, characterized in that performing the quantization to save the edges of the image, an image processing apparatus according to any one of 2, 3.

According to a fifth aspect of the present invention, a computer generates a difference image between a quantization unit that performs quantization while preserving an image edge, and a difference between the image and the quantized image quantized by the quantization unit. An image generating means; a first compressing means for compressing the quantized image quantized by the quantizing means; a second compressing means for compressing the differential image generated by the differential image generating means; and the difference The quantization means is configured to function as a control means for controlling a distance in cluster formation by the quantization means for performing the limited color processing so that the processing result by the image generation means falls within a predetermined range. as, have rows color quantization process by clustering, the control unit may set the distance to the center of gravity of each cluster generated by the clustering in advance It does not include the a exceeds the value cluster, add the centroid intermediate values range of the pixel, an image processing program.

  According to the image processing apparatus of the first aspect, in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

According to the image processing apparatus of the second aspect , when compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

According to the image processing apparatus of the third aspect , it is possible to reduce the code amount as compared with the case where the second compression unit performs the lossless compression process.

According to the image processing apparatus of the fourth aspect , it is possible to reduce the code amount as compared with the case where the original image is quantized.

According to the image processing program of the fifth aspect , in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is explanatory drawing which shows the outline | summary of the process example by this Embodiment. It is explanatory drawing which shows the outline | summary of the process example by this Embodiment. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is a conceptual module block diagram about the structural example of 3rd Embodiment. It is a notional module block diagram about the structural example of 4th Embodiment. It is a notional module block diagram about the structural example of 5th Embodiment. It is explanatory drawing which shows the example in case a difference underflows or overflows. It is a notional module block diagram about the structural example of 6th Embodiment. It is explanatory drawing which shows the process example by 6th Embodiment. It is explanatory drawing which shows the experiment example by 6th Embodiment. It is explanatory drawing which shows the experiment example by 6th Embodiment. It is explanatory drawing which shows the system configuration example using this Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise.

図１５の例に示すように、ｉ，ｊが小さい領域（図１５の例では（０，０）の係数）では、低画質（基本画像）１５３５を示す低周波となり、例えば、この部分の圧縮部分を伸長すると図１６（ａ）の例に示すような低画質な画像となる。ｉ，ｊが中程度の領域（図１５の例では（０，１）、（１，０）、・・・、（４，４）の係数）では、中画質１５４５を示すものとなり、例えば、この部分（低画質（基本画像）１５３５と中画質１５４５）の圧縮部分を伸長すると図１６（ｂ）の例に示すような中画質の画像となる。ｉ，ｊが大きい領域（図１５の例では（０，５）、（５，０）、・・・、（７，７）の係数）では、高画質１５５５を示す高周波となり、例えば、この部分（低画質（基本画像）１５３５と中画質１５４５と高画質１５５５）の圧縮部分を伸長すると図１６（ｃ）の例に示すような高画質な画像となる。なお、図１５の例に示す低画質（基本画像）１５３５、中画質１５４５、高画質１５５５からなる三角形は、前述の構造化された符号フォーマットを模式的に示したものである。もちろんのことながら、２階層以上であればよく、図１５の例に示すように、３階層であってもよいし、４階層以上であってもよい。そして、処理順番として、（０，０）、（０，１）、（１，０）、・・・のように並びが決まっている。
したがって、最初に圧縮（又は伸長）される画像（低画質（基本画像）１５３５の部分であって、例えば、図１６（ａ））は、高周波成分が含まれないことになる。エッジが含まれている文字や図形の場合は、その画像品質が課題になる。 First, before describing the present embodiment, an image processing apparatus as a premise thereof will be described with reference to examples shown in FIGS. This description is intended to facilitate understanding of the present embodiment.
There is a compression technique that realizes scalable delivery and progressive display that have a resolution and image quality determined at the time of decompression by structuring the code format.
In the technology described in Progressive-JPEG (Joint Photographic Experts Group), Patent Document 3, and the like, this is achieved using DCT coefficients, and therefore, the high-frequency component is not included in the basic image. Specifically, the DCT coefficient is generated using Equation (3) as shown in the example of FIG. For example, in JPEG, an 8 × 8 conversion coefficient is obtained.
The basic image is an initial browsing image.
The display timing (display start order) is not limited to the standard specification, but depends on the design of the expander. For example, in the case of JPEG, the smallest unit is only one DC coefficient.

As shown in the example of FIG. 15, in a region where i and j are small (coefficient of (0, 0) in the example of FIG. 15), a low frequency indicating low image quality (basic image) 1535 is obtained. When the part is expanded, a low-quality image as shown in the example of FIG. In a region where i and j are medium (coefficients (0, 1), (1, 0),..., (4, 4) in the example of FIG. 15), the medium image quality 1545 is indicated. When the compressed portion of this portion (low image quality (basic image) 1535 and medium image quality 1545) is expanded, an image with medium image quality as shown in the example of FIG. 16B is obtained. In a region where i and j are large (coefficients (0, 5), (5, 0),..., (7, 7) in the example of FIG. 15), a high frequency indicating high image quality 1555 is obtained. When the compressed portion of (low image quality (basic image) 1535, medium image quality 1545, and high image quality 1555) is expanded, a high-quality image as shown in the example of FIG. Note that the triangle composed of the low image quality (basic image) 1535, the medium image quality 1545, and the high image quality 1555 shown in the example of FIG. 15 schematically shows the structured code format described above. Of course, it is sufficient if there are two or more layers, and as shown in the example of FIG. 15, there may be three layers or four or more layers. As the processing order, the arrangement is determined as (0, 0), (0, 1), (1, 0),.
Therefore, an image (a low image quality (basic image) 1535 portion that is first compressed (or expanded), for example, FIG. 16A) does not include a high-frequency component. In the case of a character or figure that includes an edge, the image quality becomes a problem.

Further, in the techniques described in JPEG2000, Patent Document 2, and the like, this is realized using wavelet coefficients, so that the basic image does not include a high-frequency component and has the same problem as described above.
The example shown in FIG. 17 shows a low image quality (basic image) 1535 in the wavelet coefficient area 1710, a medium image quality 1545 in the wavelet coefficient area 1720, and a high image quality 1555 in the wavelet coefficient area 1730.
Then, as shown in the example of FIG. 18, only a low frequency component is repeatedly decomposed using a low-pass filter and a high-pass filter, so that the basic image does not include high frequency (edge information). In order to improve the image quality of characters containing a lot of high frequency components, information on the deep layer is required.

  In MRC (Mixed Rate Content), as shown in the example of FIG. 19, the plane (character plane (basic image) 1910, photo plane 1920) specified by tag information (plane selection information) 1930 for each coordinate. Select. Then, an expanded image 1940 is obtained as a result of the process of selecting a plane for each coordinate. For example, when first transmitted from a character plane (basic image) 1910, even if the image is first decompressed, the character quality satisfies the expected level, unlike the above-described technique. However, since it is on / off control for each coordinate, the range of image quality that can be controlled is limited. That is, the code format structured according to the image quality is not used, and it is only possible to control on / off for each coordinate. As shown in the example of FIG. 20, the example of FIG. It is not a configuration that gradually improves the image quality, such as generating a high-quality image shown in the example of FIG. 20B from a low image quality (basic image).

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
<< First Embodiment >>
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The image processing apparatus 100 according to the first embodiment compresses (encodes) an image. As shown in the example of FIG. 1, an edge preserving quantization module 110, a difference processing module 120, a compression A The image processing apparatus 150 includes a processing module 130 and a compression B processing module 140. The image processing apparatus 150 expands (decodes) the compressed image by the image processing apparatus 100. The expansion A processing module 160 and the expansion B processing module 170 And an addition processing module 180.
The outline processing contents of the image processing apparatus 100 and the image processing apparatus 150 will be described.
The target image 105 is structured by being divided into information storing characters and edges and other information, and then encoded by an encoding method (encoding method A, encoding method B) suitable for each information. Note that the encoding method A and the encoding method B may be the same. The difference is a pixel value difference. When decompressing, restore by adding on the pixel value. Of course, it can be divided into three or more planes.

  The image processing apparatus 100 receives the target image 105. Accepting the target image 105 means, for example, reading the target image 105 with a scanner, camera, etc., receiving the target image 105 from an external device via a communication line with a fax, etc., or a hard disk (those built in the computer) In addition, reading of the target image 105 stored in (including those connected via a network) is included. The image is a multivalued image (including a color image). One image may be received or a plurality of images may be received. In addition, the contents of the image include characters or figures (line drawings), and images (photos, etc.) are also included with it, for example, even if it is a document used for business, a pamphlet for advertising, etc. Good. The target image 105 (overall image 106A) is divided into a basic image 115 (low quality image 107A) and a difference image 125 (high quality image 108A) in the image processing apparatus 100.

The edge preserving quantization module 110 is connected to the difference processing module 120 and the compression A processing module 130, receives the target image 105, and passes the basic image 115 to the compression A processing module 130. The edge preserving quantization module 110 performs quantization (quantization for preserving edges) while preserving the edges of the target image 105 to generate a basic image 115. A specific example of “quantization that preserves edges” will be described later in the description of the embodiment. In the first decompression after encoding (first decompression in the stepwise decompression of a progressively compressed code), Any quantization that leaves an edge for improving the image quality of characters or graphics may be used. That is, it is desirable to quantize the gradation while leaving the resolution.
The difference processing module 120 is connected to the edge preserving quantization module 110 and the compression B processing module 140, receives the target image 105, and passes the difference image 125 to the compression B processing module 140. The difference processing module 120 generates a difference image (difference image 125) between the target image 105 and the basic image 115 quantized by the edge preserving quantization module 110.

The compression A processing module 130 is connected to the edge preserving quantization module 110, receives the basic image 115 from the edge preserving quantization module 110, and outputs a basic code 135. The compression A processing module 130 compresses the basic image 115 (quantized image) quantized by the edge preserving quantization module 110. An existing compression method may be used.
The compression B processing module 140 is connected to the difference processing module 120, receives the difference image 125 from the difference processing module 120, and outputs a difference code 145. The compression B processing module 140 compresses the difference image 125 generated by the difference processing module 120. An existing compression method may be used. Note that the same compression method as in the compression A processing module 130 may be used.

The image processing apparatus 150 receives the basic code 135 and the difference code 145 that are the processing results of the image processing apparatus 100. Here, the basic code 135 corresponds to the low-quality image 107A (low-quality image 107B), and the difference code 145 corresponds to the high-quality image 108A (high-quality image 108B). In other words, it is a structured code format.
The decompression A processing module 160 is connected to the addition processing module 180, receives the basic code 135, and passes the basic image 165 to the addition processing module 180. The decompression A processing module 160 decompresses the basic code 135 generated by the compression A processing module 130 of the image processing apparatus 100. Of course, the decompression method here corresponds to the encoding method of the compression A processing module 130. Therefore, a basic image 165 corresponding to the basic image 115 is generated. When the compression A processing module 130 performs lossless compression, the basic image 165 and the basic image 115 are the same, but when the compression A processing module 130 performs lossy compression, the basic image 165 is the same. Is not necessarily the same as the basic image 115 (in general, the basic image 165 has a smaller amount of information than the basic image 115).

The decompression B processing module 170 is connected to the addition processing module 180, receives the difference code 145, and passes the difference image 175 to the addition processing module 180. The decompression B processing module 170 decompresses the difference code 145 generated by the compression B processing module 140 of the image processing apparatus 100. Of course, the decompression method here corresponds to the encoding method of the compression B processing module 140. Therefore, a difference image 175 corresponding to the difference image 125 is generated. When the compression B processing module 140 performs lossless compression, the difference image 175 and the difference image 125 are the same. However, when the compression B processing module 140 performs lossy compression, the difference image 175 Is not necessarily the same as the difference image 125 (generally, the difference image 175 has a smaller amount of information than the difference image 125).
The addition processing module 180 is connected to the expansion A processing module 160 and the expansion B processing module 170, receives the basic image 165 from the expansion A processing module 160, receives the difference image 175 from the expansion B processing module 170, and expands the image 195. Is output. The addition processing module 180 adds the difference image 175 expanded by the expansion B processing module 170 to the basic image 165 expanded by the expansion A processing module 160. Processing corresponding to the difference processing module 120 is performed. Since progressive decompression is performed, the basic image 165 (low-quality image 107C) is first output as it is, and then an image obtained by adding the difference image 175 (high-quality image 108C) to the basic image 165 (overall image 106C). Output. To output an image is mainly to display it step by step on a display device such as a display. For example, the image is transmitted by an image transmission device such as a fax machine, or printed by a printing device such as a printer. Including writing an image to an image storage device such as an image database, storing the image in a storage medium such as a memory card, and passing the image to another information processing device.

2 and 3 are explanatory diagrams showing an outline of a processing example according to the present embodiment.
The gradation is quantized while leaving the edges of characters, etc., in the original image X205 (overall image 206A), and a basic image Y235 (low quality image 207A) is obtained. The difference image Z245 (high-quality image 208A) following the basic image Y235 is expressed by a difference in pixel values between the original image X205 and the basic image Y235 (referred to as a difference image). Note that there is a relationship of original image X = Y + Z (where X, Y, and Z are pixel values of the original image X205, the basic image Y235, and the difference image Z245).
Since a character edge (high frequency) remains in the basic image Y235, the character quality is superior to the above-described Progressive-JPEG. As shown in the example of FIG. 3, when a progressive display example is taken, characters with excellent readability are immediately displayed.
The example shown in FIG. 3A is an image obtained by decompressing the basic image Y235 (the first image displayed in the progressive display). The code amount is smaller than that shown in FIG. Can be extended.
The example shown in FIG. 3B is an image obtained by adding the result of expanding the difference image Z245 to the image obtained by expanding the basic image Y235 (the image to be displayed next in the progressive display). The image quality is higher than that shown in.

<< Second Embodiment >>
FIG. 4 is a conceptual module configuration diagram of a configuration example according to the second embodiment. In the second embodiment, a basic image with high visual sensitivity is reversibly compressed, and a low difference image is irreversibly compressed. For example, Flate (PNG) may be used for lossless compression, and JPEG may be used for lossy compression. Only the invisible information can be deleted by irreversible to reduce the code amount.
In addition, the same code | symbol is attached | subjected to the site | part of the same kind as the above-mentioned embodiment, and the overlapping description is abbreviate | omitted (hereinafter the same).
The image processing apparatus 400 includes an edge preserving quantization module 110, a difference processing module 120, a lossless compression Flate (PNG) processing module 430, and a lossy compression JPEG processing module 440.
The edge preserving quantization module 110 is connected to the difference processing module 120 and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and passes the basic image 115 to the lossless compression Flate (PNG) processing module 430.
The difference processing module 120 is connected to the edge preserving quantization module 110 and the irreversible compression JPEG processing module 440, receives the target image 105, and passes the difference image 125 to the irreversible compression JPEG processing module 440.
The lossless compression Flate (PNG) processing module 430 is connected to the edge preserving quantization module 110, receives the basic image 115 from the edge preserving quantization module 110, and outputs a basic code 435. The lossless compression Flate (PNG) processing module 430 performs lossless compression processing on the basic image 115. For example, Flate (PNG) processing is performed.
The lossy compression JPEG processing module 440 is connected to the difference processing module 120, receives the difference image 125 from the difference processing module 120, and outputs a difference code 445. The irreversible compression JPEG processing module 440 performs irreversible compression processing on the difference image 125. For example, JPEG processing is performed.

The image processing apparatus 450 includes a reversible decompression deflate (PNG) processing module 460, an irreversible decompression JPEG processing module 470, and an addition processing module 180.
The lossless decompression (PNG) processing module 460 is connected to the addition processing module 180, receives the basic code 435, and passes the basic image 165 to the addition processing module 180. The lossless decompression (PNG) processing module 460 performs decompression processing corresponding to the lossless compression Flate (PNG) processing module 430. For example, decompression processing in Flate (PNG) is performed.
The lossy decompression JPEG processing module 470 is connected to the addition processing module 180, receives the difference code 445, and passes the difference image 175 to the addition processing module 180. The irreversible decompression JPEG processing module 470 performs decompression processing corresponding to the irreversible compression JPEG processing module 440. For example, decompression processing in JPEG is performed.
The addition processing module 180 is connected to the reversible decompression deflate (PNG) processing module 460 and the irreversible decompression JPEG processing module 470, receives the basic image 165 from the reversible decompression deflate (PNG) processing module 460, and performs irreversible decompression JPEG processing. A difference image 175 is received from the module 470 and an expanded image 495 is output.
In addition, since there are many places where there is no information in the difference image, Progressive-JPEG may be used. By doing so, the number of EOBs (End Of Blocks) can be reduced, and the amount of codes can be improved.

<< Third Embodiment >>
FIG. 5 is a conceptual module configuration diagram of a configuration example according to the third embodiment. The third embodiment uses binarization processing as edge-preserving quantization. Examples of the binarization process include a simple binarization process. Specifically, if the pixel value is 128 or more, the pixel value is 255, and if it is less than 127, the pixel value is 0. Black and white characters can be stored reversibly, leaving the character readable. The binarization processing may be a method other than simple binarization, for example, floating binarization.
The image processing apparatus 500 includes a binarization processing module 510, a difference processing module 520, a lossless compression Flate (PNG) processing module 430, and a lossy compression JPEG processing module 440.
The binarization processing module 510 is connected to the difference processing module 520 and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and passes the basic image 515 to the lossless compression Flate (PNG) processing module 430. The binarization processing module 510 performs binarization processing on the target image 105 as quantization for storing image edges, and generates a basic image 515. For example, if the pixel value is 128 or more, the pixel value is 255, and if it is less than 127, the pixel value is quantized to 0.
The difference processing module 520 is connected to the binarization processing module 510 and the irreversible compression JPEG processing module 440, receives the target image 105, and passes the difference image 525 to the irreversible compression JPEG processing module 440. The difference processing module 520 calculates a difference between the target image 105 and the basic image 515 and generates a difference image 525.

The lossless compression Flate (PNG) processing module 430 is connected to the binarization processing module 510, receives the basic image 515 from the binarization processing module 510, and outputs a basic code 535.
The lossy compression JPEG processing module 440 is connected to the difference processing module 520, receives the difference image 525 from the difference processing module 520, and outputs a difference code 545.
In addition to the lossless compression Flate (PNG) processing module 430 and the lossy compression JPEG processing module 440, the compression A processing module 130 and the compression B processing module 140 may be used, respectively.
In addition, the image processing apparatus that performs the decompression process corresponding to the image processing apparatus 500 includes the decompression module A that performs the decompression process corresponding to the lossless compression Flate (PNG) processing module 430 and the irreversible process, as in the above-described embodiment. There may be a decompression module B that performs decompression processing corresponding to the compressed JPEG processing module 440 and an addition module that adds the processing results (the same applies hereinafter).

<< Fourth Embodiment >>
FIG. 6 is a conceptual module configuration diagram of an exemplary configuration according to the fourth embodiment. The fourth embodiment uses a limited color process (also called a color reduction process) as edge-preserving quantization. As the limited color processing, for example, a clustering method such as a k-average method may be used. If the color is reduced until the index can be expressed (for example, 16 colors or 256 colors), the code amount of the basic image can be greatly improved.
The image processing apparatus 600 includes a subtractive quantization clustering processing module 610, a difference processing module 620, a lossless compression flat (PNG) processing module 430, and an irreversible compression JPEG processing module 440.
The subtractive color quantization clustering processing module 610 is connected to the difference processing module 620 and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and passes the basic image 615 to the lossless compression Flate (PNG) processing module 430. . The subtractive quantization clustering processing module 610 performs limited color processing (subtractive quantization) on the target image 105 as quantization for preserving image edges, and generates a basic image 615. For example, the k-means method is used as a clustering method. Processing according to the following formulas (1) and (2) is performed.

The difference processing module 620 is connected to the subtractive color quantization clustering processing module 610 and the lossy compression JPEG processing module 440, receives the target image 105, and passes the difference image 625 to the lossy compression JPEG processing module 440. The difference processing module 620 calculates a difference between the target image 105 and the basic image 615 and generates a difference image 625.
The lossless compression Flate (PNG) processing module 430 is connected to the subtractive quantization clustering processing module 610, receives the basic image 615 from the subtractive quantization clustering processing module 610, and outputs a basic code 635.
The lossy compression JPEG processing module 440 is connected to the difference processing module 620, receives the difference image 625 from the difference processing module 620, and outputs a difference code 645.

<< Fifth Embodiment >>
FIG. 7 is a conceptual module configuration diagram of a configuration example according to the fifth embodiment. In the fifth embodiment, a pre-quantization processing module 720 that pre-quantizes a flat block (a portion that does not correspond to a character edge) that does not include an edge is provided. In the pre-quantization processing module 720, quantization is performed according to the basis vector of the latter stage (lossy compression JPEG processing module 440). More specifically, quantization is performed with an average value (DC component) of 8 × 8 blocks. The total code amount of the basic image and the difference image is improved by aligning the quantization direction with JPEG for compressing the difference image.
The image processing apparatus 700 includes a flat block determination processing module 710, a pre-quantization processing module 720, a subtractive quantization clustering processing module 610, a difference processing module 620, a reversible compression flat (PNG) processing module 430, and an irreversible compression JPEG processing module 440. have.
The flat block determination processing module 710 and the pre-quantization processing module 720 quantize the flat portion of the target image 105.
The flat block determination processing module 710 is connected to the pre-quantization processing module 720 and the subtractive quantization clustering processing module 610, receives the target image 105, passes the flat block 714 to the pre-quantization processing module 720, and performs subtractive quantization. The non-flat block 712 is passed to the clustering processing module 610. The flat block determination processing module 710 extracts a flat portion in the target image 105 and passes it to the pre-quantization processing module 720 as a flat block 714. For example, the determination process of whether or not the surface is flat may be determined by the D range or the number of colors of the block.

  The pre-quantization processing module 720 is connected to the flat block determination processing module 710 and the subtractive color quantization clustering processing module 610 and receives the flat block 714 from the flat block determination processing module 710. The pre-quantization processing module 720 performs an averaging process on, for example, an 8 × 8 pixel block for the flat block 714. By aligning the axes of JPEG and quantization, high frequency components are not left on the JPEG side. In this example, quantization is performed with a block average value (DC component) of 8 × 8 pixels so that the color reduction (rastering result) does not affect coefficients other than the DC component. Note that the quantization is not limited to 8 × 8 averaging (DC component), and quantization along the JPEG base direction is desirable in terms of the total code amount, but is not limited thereto.

The subtractive quantization clustering processing module 610 is connected to the flat block determination processing module 710, the pre-quantization processing module 720, the difference processing module 620, and the lossless compression flat (PNG) processing module 430. The non-flat block 712 is received and the basic image 615 is passed to the lossless compression Flate (PNG) processing module 430. The subtractive quantization clustering processing module 610 performs quantization for the image quantized by the pre-quantization processing module 720 to preserve the edges of the image. For example, a limited color process is performed.
The difference processing module 620 is connected to the subtractive color quantization clustering processing module 610 and the lossy compression JPEG processing module 440, receives the target image 105, and passes the difference image 625 to the lossy compression JPEG processing module 440.
The lossless compression Flate (PNG) processing module 430 is connected to the subtractive quantization clustering processing module 610, receives the basic image 615 from the subtractive quantization clustering processing module 610, and outputs a basic code 735.
The lossy compression JPEG processing module 440 is connected to the difference processing module 620, receives the difference image 625 from the difference processing module 620, and outputs a difference code 745.

<< Sixth Embodiment >>
Before describing the sixth embodiment, it will be described that the difference may underflow or overflow in the processing of the fourth embodiment.
FIG. 8 is an explanatory diagram illustrating an example in which the difference underflows or overflows. If the range of the target image 105 is 8 bits [0, 255], the difference value is [−255, 255]. Therefore, theoretically, the range may exceed the 8-bit range [−128, 127]. May be adversely affected. Of course, 8 bits is an example.
For example, if the pixel value 805 of the target image 105 is 250, and the pixel value 815 of the basic image 115 that is the processing of the edge preserving quantization module 110 is 10, the difference value that is the processing result of the difference processing module 120 Although 820 becomes 240, it becomes 127 of the range limit, and there are 125, 126, 127, etc. as irreversible results 840 that are processing results of the irreversible compression JPEG processing module 440.
Then, the decompression process for the irreversible result 840 is as follows.
・ 125 + 10 ≠ 250
・ 126 + 10 ≠ 250
127 + 10 ≠ 250
That is, the result does not return to the pixel value 250 of the original image.

FIG. 9 is a conceptual module configuration diagram of a configuration example according to the sixth embodiment. In the sixth embodiment, in order to prevent overflow and underflow, a difference value range control processing module 910 is provided to control the intensity of quantization and ensure image quality on the high image quality side. In the processing of the difference value range control processing module 910 of the sixth embodiment, control is performed so as not to be taken into a distant cluster. Specifically, a distance constraint is provided. For example, if the distance to the center of gravity of each cluster is not included in a cluster exceeding 127, the difference value can be guaranteed as [−127, 127]. In order to prevent it from being included in any cluster, for example, an intermediate value of 128 in the pixel value range may be added to the center of gravity.
The image processing apparatus 900 includes a difference value range control processing module 910, a subtractive color quantization clustering processing module 610, a difference processing module 620, a lossless compression flat (PNG) processing module 430, and a lossy compression JPEG processing module 440.
The difference value range control processing module 910 is connected to the subtractive color quantization clustering processing module 610 and receives the target image 105. The difference value range control processing module 910 controls the distance in cluster formation by the subtractive quantization clustering processing module 610 so that the processing result by the difference processing module 620 falls within a predetermined range. For example, a cluster as shown in FIG. 10 is generated. FIG. 10 is an explanatory diagram illustrating a processing example according to the sixth exemplary embodiment. An intermediate value is added to the center of gravity so that pixels not included in the cluster do not occur. Here, an example in which the color is divided into four colors (four clusters) and the distance constraint is 127 is shown.

The subtractive color quantization clustering processing module 610 is connected to the difference value range control processing module 910, the difference processing module 620, and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and receives the lossless compression Flate (PNG) processing module. A basic image 615 is passed to 430.
The lossless compression Flate (PNG) processing module 430 is connected to the subtractive quantization clustering processing module 610, receives the basic image 615 from the subtractive quantization clustering processing module 610, and outputs a basic code 935.
The lossy compression JPEG processing module 440 is connected to the difference processing module 620, receives the difference image 625 from the difference processing module 620, and outputs a difference code 945.

FIG. 11 shows an example of a decompressed image of the basic image 1107B according to the sixth embodiment and background technology (JPEG-Progressive). The code amount when the basic image 1107B was compressed was controlled to be the same. Comparing the image 1110 of the extension result by the background art and the image 1120 of the extension result by the sixth embodiment, it can be seen that the image quality of the basic image is improved by the sixth embodiment.
FIG. 12 shows an example of the code amount of the basic image 1107B according to the sixth embodiment and background technology (JPEG-Progressive). The total code amount (the total of the basic image 1107B after compression processing and the difference image 1108B) can also be improved as compared with the background art (JPEG-Progressive).

FIG. 13 is an explanatory diagram showing a system configuration example using the present embodiment.
An example in which the processing of the present embodiment is provided as a service of the cloud distribution server 1300 is shown. An example in which a compressed image is transmitted from the cloud distribution server 1300 to each user terminal 1310 will be described. The communication line between the cloud distribution server 1300 and each user terminal 1310 may be wireless, wired, or a combination thereof, for example, the Internet or an intranet as a communication infrastructure. Here, it is assumed that the target image is generated as a code format structured such as low image quality (basic image) 1335, medium image quality 1345, and high image quality 1355.
As an example of distribution according to the line speed, a user terminal 1310A connected to a low-speed line transmits a code of only low image quality (basic image) 1335 and is connected to a medium-speed line. The low quality (basic image) 1335 and medium quality 1345 codes are transmitted to 1310B, and the low quality (basic image) 1335, medium quality 1345, and high quality 1355 are transmitted to the user terminal 1310C connected to the high-speed line. Is transmitted.
As an example of delivery in the image quality desired by the user, a code of only low image quality (basic image) 1335 is transmitted to the user terminal 1320A of the user who desires low image quality, and low image quality is transmitted to the user terminal 1320B of the user who desires medium image quality Codes of (basic image) 1335 and medium image quality 1345 are transmitted, and codes of low image quality (basic image) 1335, medium image quality 1345 and image quality 1355 are transmitted to the user terminal 1320C who desires high image quality.
As an example of delivery in progressive display (an example in which image quality improves with time), a code with only low image quality (basic image) 1335 is transmitted at time (t = 1), and then (t = 2) Codes of (basic image) 1335 and medium image quality 1345 are transmitted, and then (t = 3), codes of low image quality (basic image) 1335, medium image quality 1345, and high image quality 1355 are transmitted. As a result, at the time (t = 1), the user terminal 1330 displays an image that is vague (“hazy” when compared to time (t = 2 or t = 3)), and the time (t = In 2), a clear image is displayed, and at time (t = 3), a clearer image is displayed.

  A hardware configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 14 is configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 1417 such as a scanner and a data output unit 1418 such as a printer.

  A CPU (Central Processing Unit) 1401 includes various modules described in the above-described embodiments, that is, the edge preserving quantization module 110, the difference processing module 120, the compression A processing module 130, the compression B processing module 140, and the decompression A processing. Module 160, decompression B processing module 170, addition processing module 180, lossless compression Flate (PNG) processing module 430, lossy compression JPEG processing module 440, lossless decompression Deflate (PNG) processing module 460, lossy decompression JPEG processing module 470, Binarization processing module 510, difference processing module 520, subtractive color quantization clustering processing module 610, difference processing module 620, flat block determination processing module 7 10, a control unit that executes processing according to a computer program that describes an execution sequence of each module such as a pre-quantization processing module 720 and a difference value range control processing module 910.

  A ROM (Read Only Memory) 1402 stores programs used by the CPU 1401, calculation parameters, and the like. A RAM (Random Access Memory) 1403 stores programs used in the execution of the CPU 1401, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 1404 including a CPU bus or the like.

  The host bus 1404 is connected to an external bus 1406 such as a peripheral component interconnect / interface (PCI) bus via a bridge 1405.

  A keyboard 1408 and a pointing device 1409 such as a mouse are input devices operated by an operator. The display 1410 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 1411 includes a hard disk (may be a flash memory or the like), drives the hard disk, and records or reproduces a program executed by the CPU 1401 and information. The hard disk stores the target image 105, the basic code 135, the differential code 145, the basic image 115, the differential image 125, the basic image 165, the differential image 175, and the like. Further, various other data, various computer programs, and the like are stored.

  The drive 1412 reads out data or a program recorded in a removable recording medium 1413 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out as an interface 1407 and an external bus 1406. , The bridge 1405, and the RAM 1403 connected via the host bus 1404. The removable recording medium 1413 can also be used as a data recording area similar to a hard disk.

  The connection port 1414 is a port for connecting the external connection device 1415 and has a connection unit such as USB and IEEE1394. The connection port 1414 is connected to the CPU 1401 and the like via the interface 1407, the external bus 1406, the bridge 1405, the host bus 1404, and the like. A communication unit 1416 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1417 is a scanner, for example, and executes document reading processing. The data output unit 1418 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus illustrated in FIG. 14 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 14, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 14 may be connected to each other via communication lines so as to cooperate with each other. In particular, in addition to personal computers, portable information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc.), information appliances, copiers, fax machines, scanners, printers, multifunction devices (scanners, printers, copiers) Or an image processing apparatus having two or more functions such as a fax machine).

  Note that the above-described various embodiments may be combined (for example, adding or replacing a module in one embodiment in another embodiment), and processing contents of each module The technique described in the background art may be employed.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 105 ... Object image 110 ... Edge preservation | save quantization module 115 ... Basic image 120 ... Difference processing module 125 ... Difference image 130 ... Compression A processing module 135 ... Basic code 140 ... Compression B processing module 145 ... Difference code 150 Image processing device 160 ... Decompression A processing module 165 ... Basic image 170 ... Decompression B processing module 175 ... Difference image 180 ... Addition processing module 195 ... Decompression image 400 ... Image processing device 430 ... Reversible compression Flate (PNG) processing module 435 ... Basic code 440 ... lossy compression JPEG processing module 445 ... difference code 450 ... image processing device 460 ... lossless decompression deflate (PNG) processing module 470 ... lossy decompression JPEG processing module 495 ... decompressed image 500 ... image Processing device 510 ... Binarization processing module 515 ... Basic image 520 ... Difference processing module 525 ... Difference image 535 ... Basic code 545 ... Difference code 600 ... Image processing device 610 ... Subtractive color quantization clustering processing module 615 ... Basic image 620 ... Difference Processing module 625 ... difference image 635 ... basic code 645 ... difference code 700 ... image processing device 710 ... flat block determination processing module 712 ... non-flat block 714 ... flat block 720 ... pre-quantization processing module 735 ... basic code 745 ... difference code 900 ... Image processing device 910 ... Difference value range control processing module 935 ... Basic code 945 ... Difference code 1300 ... Cloud distribution server 1310 ... User terminal 1320 ... User terminal 1330 ... User terminal

Claims (5)

A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
First compression means for compressing the quantized image quantized by the quantization means;
Second compression means for compressing the difference image generated by the difference image generation means ;
Control means for controlling the distance in cluster formation by the quantization means for performing the limited color processing so that the processing result by the difference image generation means falls within a predetermined range ;
The quantization means, a quantization have rows color quantization process by clustering,
The control means prevents the distance to the center of gravity of each cluster generated by the clustering from being included in a cluster exceeding a predetermined value, and adds an intermediate value of the pixel value range to the center of gravity.
An image processing apparatus. A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
First compression means for compressing the quantized image quantized by the quantization means;
Second compression means for compressing the difference image generated by the difference image generation means;
Control means for controlling the distance in the cluster formation by the quantization means for performing the limited color processing so that the processing result by the difference image generation means falls within a predetermined range;
First decompression means for decompressing the code generated by the first compression means;
Second decompression means for decompressing the code generated by the second compression means;
Adding means for adding the image expanded by the second expansion means to the image expanded by the first expansion means;
The quantization means, a quantization have rows color quantization process by clustering,
The control means prevents the distance to the center of gravity of each cluster generated by the clustering from being included in a cluster exceeding a predetermined value, and adds an intermediate value of the pixel value range to the center of gravity.
An image processing apparatus. The first compression means performs a reversible compression process,
The image processing apparatus according to claim 1, wherein the second compression unit performs irreversible compression processing. Second quantization means for quantizing a flat portion of the image,
The said quantization means performs the quantization which preserve | saves the edge of this image for the 2nd quantization image quantized by the said 2nd quantization means as object. The image processing apparatus according to any one of the above. Computer
A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
First compression means for compressing the quantized image quantized by the quantization means;
Second compression means for compressing the difference image generated by the difference image generation means ;
Functioning as control means for controlling the distance in cluster formation by the quantizing means for performing the limited color processing so that the processing result by the difference image generating means falls within a predetermined range ;
The quantization means, a quantization have rows color quantization process by clustering,
The control means prevents the distance to the center of gravity of each cluster generated by the clustering from being included in a cluster exceeding a predetermined value, and adds an intermediate value of the pixel value range to the center of gravity.
Image processing program.

Images