JP2017022652A - Image processing device and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that is configured to further improve image quality of an image obtained after first decompression, in comparison with Progressive-JPEG, and to suppress degradation in image quality even if a difference image does not fall within a predetermined range of values.SOLUTION: Quantization means of an image forming device quantizes an image, while preserving an edge of the image. Difference image generation means generates a difference image as a difference between the image and a quantized image quantized by the quantization means. Division means, if the image generated by the difference image generation means does not fall within a predetermined range of values, divides the image into a first image and a second image to fall within the range of values. First compression means compresses the quantized image quantized by the quantization means, and second compression means compresses the first image and the second image divided by the division means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Patent Document 1 transmits scalable video according to priority, solves the problem of optimization selection and transmission of a bit stream of scalable video, and finally improves the quality of an image reconstructed from the bit stream The priority is rearranged for the layered bitstream after the SVC encoding of each GOP of the scalable video, and the package and the Raptor encoding are performed for the SVC layered bitstream after the rearrangement. And transmitting the encoded scalable video bit stream to the scalable video receiver via the transmission channel.

  In Patent Document 2, it is an object to compress and encode image data with a small amount of computation and at high speed, and to obtain a compressed image that matches the target image quality while removing noise. In addition, the data is truncated so that a desired noise removal effect can be obtained with respect to the code sequence subjected to the bit shift, and the data is truncated sequentially from the rightmost bit, for example, from bit data of number 0 of VHL4. If the bit data up to YHH1 is deleted in order, and the target noise removal effect can be obtained by truncating the YHH5 number 0 bit data in the downward direction, the data of the corresponding scattered dot portion is truncated. If the desired noise removal effect cannot be obtained even if the data of YHH1 is rounded down, the bit data of number 0 of VLL4 is sequentially moved downward. It is disclosed that slide into dividing.

  Patent Document 3 provides a video transmission terminal and a video transmission terminal capable of realizing the scalability of encoded data obtained by object video encoding without degrading the quality of an important part of a video. The priority area is set from the important part of the area extraction data extracted by the area extraction section for the subject of the digital video obtained from the video source, and the object video encoding section is encoded for the set priority area. Thus, it is disclosed that scalability is realized for encoded data, and even if the encoded data is reproduced, important portions can be displayed without degrading the quality.

JP 2011-147120 A JP 2007-104645 A JP 2001-066952 A

As a method for compressing an image, there is Progressive-JPEG or the like. In these techniques, a rough low-frequency image is first displayed over the entire image, and a fine high-frequency image is gradually displayed. Therefore, the first basic image does not include a high-frequency component.
Therefore, since the first image does not contain a high frequency component, an edge is lost.
For example, a phenomenon occurs in which the recognition rate of a character image decreases in the first image.
The present invention improves the image quality of the image after the first decompression compared to Progressive-JPEG, and suppresses the deterioration of the image quality even when the difference image does not fall within a predetermined range. An object of the present invention is to provide an image processing apparatus and an image processing program.

The gist of the present invention for achieving the object lies in the inventions of the following items.
The invention according to claim 1 is a difference image generating means for generating a difference image between a quantization means for performing quantization while preserving an edge of an image and the image and the quantized image quantized by the quantization means. A dividing unit that divides the image into a first image and a second image that fall within the range when the image generated by the difference image generation unit does not fall within a predetermined range; and the quantization Comprising: a first compression means for compressing the quantized image quantized by the means; and a second compression means for compressing the first image and the second image divided by the division means. An image processing apparatus.

  According to a second aspect of the present invention, there is provided a first decompression unit for decompressing a code generated by the first compression unit of the image processing apparatus of the first aspect, and a second compression unit of the image processing apparatus of the first aspect. Second decompression means for decompressing the first code and the second code, the first image decompressed by the second decompression means, and the second image decompressed by the first decompression means An image processing apparatus comprising addition means for adding the images.

  According to a third aspect of the present invention, there is provided a quantization means for performing quantization while preserving an edge of an image, and a difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means. A dividing unit that divides the image into a first image and a second image that fall within the range when the image generated by the difference image generation unit does not fall within a predetermined range; and the quantization First compression means for compressing the quantized image quantized by the means, second compression means for compressing the first image and the second image divided by the dividing means, and the first compression First decompression means for decompressing the code generated by the means, second decompression means for decompressing the first code and the second code generated by the second compression means, and the first decompression The image expanded by the means An image processing apparatus characterized by comprising adding means for adding the first and second images that have been extended by two expansion means.

  4. The image processing according to claim 1, wherein the first compression unit performs a reversible compression process, and the second compression unit performs a lossy compression process. Device.

  A fifth aspect of the present invention is the image processing apparatus according to any one of the first, third, and fourth aspects, wherein the quantization unit performs binarization processing as quantization.

  A sixth aspect of the present invention is the image processing apparatus according to any one of the first, third, and fourth aspects, wherein the quantization unit performs a limited color processing as quantization.

  The invention of claim 7 further comprises second quantization means for quantizing a flat portion of the image, wherein the quantization means is a second quantized image quantized by the second quantization means. The image processing apparatus according to claim 1, wherein quantization for preserving an edge of the image is performed.

  The invention of claim 8 further comprises a control means for controlling a distance in cluster formation by the quantizing means for performing the limited color processing so that the processing result by the difference image generating means falls within a predetermined range. An image processing apparatus according to claim 6.

  According to a ninth aspect of the present invention, a computer generates a difference image between a quantization unit that performs quantization while preserving an image edge, and a difference between the image and a quantized image quantized by the quantization unit. An image generating unit; and a dividing unit that divides the image into a first image and a second image that fall within the predetermined range when the image generated by the difference image generating unit does not fall within a predetermined range. To function as a first compression unit that compresses the quantized image quantized by the quantization unit, and a second compression unit that compresses the first image and the second image divided by the division unit. This is an image processing program.

  The invention according to claim 10 is generated by the first decompressing means for decompressing the code generated by the first compressing means of the image processing apparatus of claim 1, and by the second compressing means of the image processing apparatus of claim 1. And a display unit for displaying the image expanded by the first expansion unit in advance with respect to the image added by the addition unit. An image processing apparatus.

  According to an eleventh aspect of the present invention, there is provided a computer, a first decompression unit for decompressing the code generated by the first compression unit of the image processing program of the ninth aspect, and a second compression of the image processing program of the ninth aspect. Second decompression means for decompressing the first code and the second code generated by the means, and a first image decompressed by the second decompression means to an image decompressed by the first decompression means And an image processing program for functioning as an adding means for adding the second image.

  According to a twelfth aspect of the present invention, a computer generates a difference image between a quantization unit that performs quantization while preserving an image edge, and a difference between the image and the quantized image quantized by the quantization unit. An image generating unit; and a dividing unit that divides the image into a first image and a second image that fall within the predetermined range when the image generated by the difference image generating unit does not fall within a predetermined range. A first compression means for compressing the quantized image quantized by the quantization means; a second compression means for compressing the first image and the second image divided by the dividing means; A first decompression unit for decompressing the code generated by the first compression unit; a second decompression unit for decompressing the first code and the second code generated by the second compression unit; Stretched by 1 stretching means An image processing program for functioning as a first image and adding means for adding the second image that has been extended by the second expansion means to image.

  According to a thirteenth aspect of the present invention, the computer includes a first decompression unit that decompresses the code generated by the first compression unit of the image processing program of the ninth aspect, and a second compression of the image processing program of the ninth aspect. A second decompressing means for decompressing the code generated by the means; and an image decompressed by the first decompressing means for causing the image added by the adding means to function as a display means for displaying first. This is an image processing program.

  According to the image processing apparatus of claim 1, in the case of compressing an image, the image quality after the first decompression can be improved as compared with Progressive-JPEG, and the difference image is determined in advance. Even when the value does not fall within the range, it is possible to suppress deterioration in image quality.

  According to the image processing apparatus of the second aspect, in the case of decompressing a compressed image, the image quality of the first image can be improved compared to Progressive-JPEG, and the difference image is in a predetermined range. Even if it does not fit, deterioration of image quality can be suppressed.

  According to the image processing apparatus of the third aspect, in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

  According to the image processing apparatus of the fourth aspect, it is possible to reduce the code amount as compared with the case where the second compression unit performs the lossless compression process.

  According to the image processing apparatus of the fifth aspect, in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

  According to the image processing apparatus of the sixth aspect, in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

  According to the image processing device of the seventh aspect, it is possible to reduce the code amount as compared with the case where the original image is quantized.

  According to the image processing apparatus of the eighth aspect, it is possible to improve the image quality as compared with the case where the difference underflows or overflows.

  According to the image processing program of claim 9, in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG, and the difference image is determined in advance. Even when the value does not fall within the range, it is possible to suppress deterioration in image quality.

  According to the image processing apparatus of the tenth aspect, it is possible to improve the image quality of an image to be displayed earlier as compared with Progressive-JPEG.

  According to the image processing program of the eleventh aspect, in the case of decompressing a compressed image, the image quality of the first image can be improved compared to Progressive-JPEG, and the difference image is in a predetermined range. Even if it does not fit, deterioration of image quality can be suppressed.

  According to the image processing program of the twelfth aspect, when the image is compressed, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.

  According to the image processing program of the thirteenth aspect, it is possible to improve the image quality of an image to be displayed earlier as compared with Progressive-JPEG.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is explanatory drawing which shows the outline | summary of the process example by this Embodiment. It is explanatory drawing which shows the outline | summary of the process example by this Embodiment. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is a conceptual module block diagram about the structural example of 3rd Embodiment. It is a notional module block diagram about the structural example of 4th Embodiment. It is a notional module block diagram about the structural example of 5th Embodiment. It is explanatory drawing which shows the example in case a difference underflows or overflows. It is a notional module block diagram about the structural example of 6th Embodiment. It is explanatory drawing which shows the process example by 6th Embodiment. It is explanatory drawing which shows the experiment example by 6th Embodiment. It is explanatory drawing which shows the experiment example by 6th Embodiment. It is explanatory drawing which shows the system configuration example using this Embodiment. It is a notional module block diagram about the structural example of 7th Embodiment. It is a flowchart which shows the process example by 7th Embodiment. It is a flowchart which shows the process example by 7th Embodiment. It is a flowchart which shows the process example by 7th Embodiment. It is a flowchart which shows the process example by 7th Embodiment. It is a flowchart which shows the process example by 7th Embodiment. It is explanatory drawing which shows the process example by 7th Embodiment. It is explanatory drawing which shows the process example by 7th Embodiment. It is explanatory drawing which shows the process example by 7th Embodiment. It is explanatory drawing which shows the process example by 7th Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise. It is explanatory drawing which shows the process example of the image processing apparatus used as a premise.

図２５の例に示すように、ｉ，ｊが小さい領域（図２５の例では（０，０）の係数）では、低画質（基本画像）２５３５を示す低周波となり、例えば、この部分の圧縮部分を伸長すると図２６（ａ）の例に示すような低画質な画像となる。ｉ，ｊが中程度の領域（図２５の例では（０，１）、（１，０）、・・・、（４，４）の係数）では、中画質２５４５を示すものとなり、例えば、この部分（低画質（基本画像）２５３５と中画質２５４５）の圧縮部分を伸長すると図２６（ｂ）の例に示すような中画質の画像となる。ｉ，ｊが大きい領域（図２５の例では（０，５）、（５，０）、・・・、（７，７）の係数）では、高画質２５５５を示す高周波となり、例えば、この部分（低画質（基本画像）２５３５と中画質２５４５と高画質２５５５）の圧縮部分を伸長すると図２６（ｃ）の例に示すような高画質な画像となる。なお、図２５の例に示す低画質（基本画像）２５３５、中画質２５４５、高画質２５５５からなる三角形は、前述の構造化された符号フォーマットを模式的に示したものである。もちろんのことながら、２階層以上であればよく、図２５の例に示すように、３階層であってもよいし、４階層以上であってもよい。そして、処理順番として、（０，０）、（０，１）、（１，０）、・・・のように並びが決まっている。
したがって、最初に圧縮（又は伸長）される画像（低画質（基本画像）２５３５の部分であって、例えば、図２６（ａ））は、高周波成分が含まれないことになる。エッジが含まれている文字や図形の場合は、その画像品質が課題になる。 First, before describing the present embodiment, an image processing apparatus as a premise thereof will be described with reference to examples shown in FIGS. This description is intended to facilitate understanding of the present embodiment.
There is a compression technique that realizes scalable delivery and progressive display that have a resolution and image quality determined at the time of decompression by structuring the code format.
In the technology described in Progressive-JPEG (Joint Photographic Experts Group), Patent Document 3, and the like, this is achieved using DCT coefficients, and therefore, the high-frequency component is not included in the basic image. Specifically, DCT coefficients are generated using Expression (3) as shown in the example of FIG. For example, in JPEG, an 8 × 8 conversion coefficient is obtained.
The basic image is an initial browsing image.
The display timing (display start order) is not limited to the standard specification, but depends on the design of the expander. For example, in the case of JPEG, the smallest unit is only one DC coefficient.

As shown in the example of FIG. 25, in a region where i and j are small (coefficient of (0, 0) in the example of FIG. 25), a low frequency indicating low image quality (basic image) 2535 is obtained. When the part is expanded, a low-quality image as shown in the example of FIG. In a region where i and j are medium (coefficients (0, 1), (1, 0),..., (4, 4) in the example of FIG. 25), the medium image quality 2545 is indicated. When the compressed portion of this portion (low image quality (basic image) 2535 and medium image quality 2545) is expanded, an image with medium image quality as shown in the example of FIG. In a region where i and j are large (coefficients (0, 5), (5, 0),..., (7, 7) in the example of FIG. 25), a high frequency indicating high image quality 2555 is obtained. When the compression portion of (low image quality (basic image) 2535, medium image quality 2545, and high image quality 2555) is expanded, a high image quality as shown in the example of FIG. Note that the triangle composed of the low image quality (basic image) 2535, the medium image quality 2545, and the high image quality 2555 shown in the example of FIG. 25 schematically shows the structured code format described above. Of course, it is sufficient if there are two or more layers, and as shown in the example of FIG. 25, there may be three layers or four or more layers. As the processing order, the arrangement is determined as (0, 0), (0, 1), (1, 0),.
Therefore, an image (low image quality (basic image) 2535 portion, for example, FIG. 26A) to be compressed (or expanded) first does not include a high frequency component. In the case of a character or figure that includes an edge, the image quality becomes a problem.

Further, in the techniques described in JPEG2000, Patent Document 2, and the like, this is realized using wavelet coefficients, so that the basic image does not include a high-frequency component and has the same problem as described above.
In the example illustrated in FIG. 27, the wavelet coefficient region 2710 indicates low image quality (basic image) 2535, the wavelet coefficient region 2720 indicates medium image quality 2545, and the wavelet coefficient region 2730 indicates high image quality 2555.
As shown in the example of FIG. 28, only the low frequency component is repeatedly decomposed using the low pass filter and the high pass filter, so that the basic image does not include high frequency (edge information). In order to improve the image quality of characters containing a lot of high frequency components, information on the deep layer is required.

  In MRC (Mixed Rate Content), as shown in the example of FIG. 29, planes (character plane (basic image) 2910, photo plane 2920) designated by tag information (plane selection information) 2930 for each coordinate. Select. Then, an expanded image 2940 is obtained as a result of the process of selecting a plane for each coordinate. For example, when the image is first transmitted from the character plane (basic image) 2910, even if the image is first decompressed, the character quality satisfies the expected level unlike the above-described technique. However, since it is on / off control for each coordinate, the range of image quality that can be controlled is limited. That is, the code format structured according to the image quality is not used, and it is only possible to control on / off for each coordinate. As shown in the example of FIG. 30, the example of FIG. It is not a configuration for gradually improving the image quality, such as generating a high-quality image shown in the example of FIG. 30B from a low image quality (basic image).

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
<< First Embodiment >>
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, the values may be different from each other, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The image processing apparatus 100 according to the first embodiment compresses (encodes) an image. As shown in the example of FIG. 1, an edge preserving quantization module 110, a difference processing module 120, a compression A The image processing apparatus 150 includes a processing module 130 and a compression B processing module 140. The image processing apparatus 150 expands (decodes) the compressed image by the image processing apparatus 100. The expansion A processing module 160 and the expansion B processing module 170 And an addition processing module 180.
The outline processing contents of the image processing apparatus 100 and the image processing apparatus 150 will be described.
The target image 105 is structured by being divided into information storing characters and edges and other information, and then encoded by an encoding method (encoding method A, encoding method B) suitable for each information. Note that the encoding method A and the encoding method B may be the same. The difference is a pixel value difference. When decompressing, restore by adding on the pixel value. Of course, it can be divided into three or more planes.

  The image processing apparatus 100 receives the target image 105. Accepting the target image 105 means, for example, reading the target image 105 with a scanner, a camera, etc., receiving the target image 105 from an external device via a communication line with a fax machine, etc. In addition, reading of the target image 105 stored in (including those connected via a network) is included. The image is a multivalued image (including a color image). One image may be received or a plurality of images may be received. In addition, the contents of the image include characters or figures (line drawings), and images (photos, etc.) are also included along with them. For example, even for documents used in business, brochures for advertisements, etc. Good. The target image 105 (overall image 106A) is divided into a basic image 115 (low quality image 107A) and a difference image 125 (high quality image 108A) in the image processing apparatus 100.

The edge preserving quantization module 110 is connected to the difference processing module 120 and the compression A processing module 130, receives the target image 105, and passes the basic image 115 to the compression A processing module 130. The edge preserving quantization module 110 performs quantization (quantization for preserving edges) while preserving the edges of the target image 105 to generate a basic image 115. A specific example of “quantization that preserves edges” will be described later in the description of the embodiment. In the first decompression after encoding (first decompression in the stepwise decompression of a progressively compressed code), Any quantization that leaves an edge for improving the image quality of characters or graphics may be used. That is, it is desirable to quantize the gradation while leaving the resolution.
The difference processing module 120 is connected to the edge preserving quantization module 110 and the compression B processing module 140, receives the target image 105, and passes the difference image 125 to the compression B processing module 140. The difference processing module 120 generates a difference image (difference image 125) between the target image 105 and the basic image 115 quantized by the edge preserving quantization module 110.

The compression A processing module 130 is connected to the edge preserving quantization module 110, receives the basic image 115 from the edge preserving quantization module 110, and outputs a basic code 135. The compression A processing module 130 compresses the basic image 115 (quantized image) quantized by the edge preserving quantization module 110. An existing compression method may be used.
The compression B processing module 140 is connected to the difference processing module 120, receives the difference image 125 from the difference processing module 120, and outputs a difference code 145. The compression B processing module 140 compresses the difference image 125 generated by the difference processing module 120. An existing compression method may be used. Note that the same compression method as in the compression A processing module 130 may be used.

The image processing apparatus 150 receives the basic code 135 and the difference code 145 that are the processing results of the image processing apparatus 100. Here, the basic code 135 corresponds to the low-quality image 107A (low-quality image 107B), and the difference code 145 corresponds to the high-quality image 108A (high-quality image 108B). In other words, it is a structured code format.
The decompression A processing module 160 is connected to the addition processing module 180, receives the basic code 135, and passes the basic image 165 to the addition processing module 180. The decompression A processing module 160 decompresses the basic code 135 generated by the compression A processing module 130 of the image processing apparatus 100. Of course, the decompression method here corresponds to the encoding method of the compression A processing module 130. Therefore, a basic image 165 corresponding to the basic image 115 is generated. When the compression A processing module 130 performs lossless compression, the basic image 165 and the basic image 115 are the same, but when the compression A processing module 130 performs lossy compression, the basic image 165 is the same. Is not necessarily the same as the basic image 115 (in general, the basic image 165 has a smaller amount of information than the basic image 115).

The decompression B processing module 170 is connected to the addition processing module 180, receives the difference code 145, and passes the difference image 175 to the addition processing module 180. The decompression B processing module 170 decompresses the difference code 145 generated by the compression B processing module 140 of the image processing apparatus 100. Of course, the decompression method here corresponds to the encoding method of the compression B processing module 140. Therefore, a difference image 175 corresponding to the difference image 125 is generated. When the compression B processing module 140 performs lossless compression, the difference image 175 and the difference image 125 are the same. However, when the compression B processing module 140 performs lossy compression, the difference image 175 Is not necessarily the same as the difference image 125 (generally, the difference image 175 has a smaller amount of information than the difference image 125).
The addition processing module 180 is connected to the expansion A processing module 160 and the expansion B processing module 170, receives the basic image 165 from the expansion A processing module 160, receives the difference image 175 from the expansion B processing module 170, and expands the image 195. Is output. The addition processing module 180 adds the difference image 175 expanded by the expansion B processing module 170 to the basic image 165 expanded by the expansion A processing module 160. Processing corresponding to the difference processing module 120 is performed. Since progressive decompression is performed, the basic image 165 (low-quality image 107C) is first output as it is, and then an image obtained by adding the difference image 175 (high-quality image 108C) to the basic image 165 (overall image 106C). Output. To output an image is mainly to display it step by step on a display device such as a display. For example, the image is transmitted by an image transmission device such as a fax machine, or printed by a printing device such as a printer. Including writing an image to an image storage device such as an image database, storing the image in a storage medium such as a memory card, and passing the image to another image processing device.

2 and 3 are explanatory diagrams showing an outline of a processing example according to the present embodiment.
The gradation is quantized while leaving the edges of characters, etc., in the original image X205 (overall image 206A), and a basic image Y235 (low quality image 207A) is obtained. The difference image Z245 (high-quality image 208A) following the basic image Y235 is expressed by a difference in pixel values between the original image X205 and the basic image Y235 (referred to as a difference image). Note that there is a relationship of original image X = Y + Z (where X, Y, and Z are pixel values of the original image X205, the basic image Y235, and the difference image Z245).
Since a character edge (high frequency) remains in the basic image Y235, the character quality is superior to the above-described Progressive-JPEG. As shown in the example of FIG. 3, when a progressive display example is taken, characters with excellent readability are immediately displayed.
The example shown in FIG. 3A is an image obtained by decompressing the basic image Y235 (the first image displayed in the progressive display). The code amount is smaller than that shown in FIG. Can be extended.
The example shown in FIG. 3B is an image obtained by adding the result of expanding the difference image Z245 to the image obtained by expanding the basic image Y235 (the image to be displayed next in the progressive display). The image quality is higher than that shown in.

<< Second Embodiment >>
FIG. 4 is a conceptual module configuration diagram of a configuration example according to the second embodiment. In the second embodiment, a basic image with high visual sensitivity is reversibly compressed, and a low difference image is irreversibly compressed. For example, Flate (PNG) may be used for lossless compression, and JPEG may be used for lossy compression. Only the invisible information can be deleted by irreversible to reduce the code amount.
In addition, the same code | symbol is attached | subjected to the site | part of the same kind as the above-mentioned embodiment, and the overlapping description is abbreviate | omitted (hereinafter the same).
The image processing apparatus 400 includes an edge preserving quantization module 110, a difference processing module 120, a lossless compression Flate (PNG) processing module 430, and a lossy compression JPEG processing module 440.
The edge preserving quantization module 110 is connected to the difference processing module 120 and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and passes the basic image 115 to the lossless compression Flate (PNG) processing module 430.
The difference processing module 120 is connected to the edge preserving quantization module 110 and the irreversible compression JPEG processing module 440, receives the target image 105, and passes the difference image 125 to the irreversible compression JPEG processing module 440.
The lossless compression Flate (PNG) processing module 430 is connected to the edge preserving quantization module 110, receives the basic image 115 from the edge preserving quantization module 110, and outputs a basic code 435. The lossless compression Flate (PNG) processing module 430 performs lossless compression processing on the basic image 115. For example, Flate (PNG) processing is performed.
The lossy compression JPEG processing module 440 is connected to the difference processing module 120, receives the difference image 125 from the difference processing module 120, and outputs a difference code 445. The irreversible compression JPEG processing module 440 performs irreversible compression processing on the difference image 125. For example, JPEG processing is performed.

The image processing apparatus 450 includes a reversible decompression deflate (PNG) processing module 460, an irreversible decompression JPEG processing module 470, and an addition processing module 180.
The lossless decompression (PNG) processing module 460 is connected to the addition processing module 180, receives the basic code 435, and passes the basic image 165 to the addition processing module 180. The lossless decompression (PNG) processing module 460 performs decompression processing corresponding to the lossless compression Flate (PNG) processing module 430. For example, decompression processing in Flate (PNG) is performed.
The lossy decompression JPEG processing module 470 is connected to the addition processing module 180, receives the difference code 445, and passes the difference image 175 to the addition processing module 180. The irreversible decompression JPEG processing module 470 performs decompression processing corresponding to the irreversible compression JPEG processing module 440. For example, decompression processing in JPEG is performed.
The addition processing module 180 is connected to the reversible decompression deflate (PNG) processing module 460 and the irreversible decompression JPEG processing module 470, receives the basic image 165 from the reversible decompression deflate (PNG) processing module 460, and performs irreversible decompression JPEG processing. A difference image 175 is received from the module 470 and an expanded image 495 is output.
In addition, since there are many places where there is no information in the difference image, Progressive-JPEG may be used. By doing so, the number of EOBs (End Of Blocks) can be reduced, and the amount of codes can be improved.

<< Third Embodiment >>
FIG. 5 is a conceptual module configuration diagram of a configuration example according to the third embodiment. The third embodiment uses binarization processing as edge-preserving quantization. Examples of the binarization process include a simple binarization process. Specifically, if the pixel value is 128 or more, the pixel value is 255, and if it is less than 127, the pixel value is 0. Black and white characters can be stored reversibly, leaving the character readable. The binarization processing may be a method other than simple binarization, for example, floating binarization.
The image processing apparatus 500 includes a binarization processing module 510, a difference processing module 520, a lossless compression Flate (PNG) processing module 430, and a lossy compression JPEG processing module 440.
The binarization processing module 510 is connected to the difference processing module 520 and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and passes the basic image 515 to the lossless compression Flate (PNG) processing module 430. The binarization processing module 510 performs binarization processing on the target image 105 as quantization for storing image edges, and generates a basic image 515. For example, if the pixel value is 128 or more, the pixel value is 255, and if it is less than 127, the pixel value is quantized to 0.
The difference processing module 520 is connected to the binarization processing module 510 and the irreversible compression JPEG processing module 440, receives the target image 105, and passes the difference image 525 to the irreversible compression JPEG processing module 440. The difference processing module 520 calculates a difference between the target image 105 and the basic image 515 and generates a difference image 525.

The lossless compression Flate (PNG) processing module 430 is connected to the binarization processing module 510, receives the basic image 515 from the binarization processing module 510, and outputs a basic code 535.
The lossy compression JPEG processing module 440 is connected to the difference processing module 520, receives the difference image 525 from the difference processing module 520, and outputs a difference code 545.
In addition to the lossless compression Flate (PNG) processing module 430 and the lossy compression JPEG processing module 440, the compression A processing module 130 and the compression B processing module 140 may be used, respectively.
In addition, the image processing apparatus that performs the decompression process corresponding to the image processing apparatus 500 includes the decompression module A that performs the decompression process corresponding to the lossless compression Flate (PNG) processing module 430 and the irreversible process, as in the above-described embodiment. There may be a decompression module B that performs decompression processing corresponding to the compressed JPEG processing module 440 and an addition module that adds the processing results (the same applies hereinafter).

<< Fourth Embodiment >>
FIG. 6 is a conceptual module configuration diagram of an exemplary configuration according to the fourth embodiment. The fourth embodiment uses a limited color process (also called a color reduction process) as edge-preserving quantization. As the limited color processing, for example, a clustering method such as a k-average method may be used. If the color is reduced until the index can be expressed (for example, 16 colors or 256 colors), the code amount of the basic image can be greatly improved.
The image processing apparatus 600 includes a subtractive quantization clustering processing module 610, a difference processing module 620, a lossless compression flat (PNG) processing module 430, and an irreversible compression JPEG processing module 440.
The subtractive color quantization clustering processing module 610 is connected to the difference processing module 620 and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and passes the basic image 615 to the lossless compression Flate (PNG) processing module 430. . The subtractive quantization clustering processing module 610 performs limited color processing (subtractive quantization) on the target image 105 as quantization for preserving image edges, and generates a basic image 615. For example, the k-means method is used as a clustering method. Processing according to the following formulas (1) and (2) is performed.

The difference processing module 620 is connected to the subtractive color quantization clustering processing module 610 and the lossy compression JPEG processing module 440, receives the target image 105, and passes the difference image 625 to the lossy compression JPEG processing module 440. The difference processing module 620 calculates a difference between the target image 105 and the basic image 615 and generates a difference image 625.
The lossless compression Flate (PNG) processing module 430 is connected to the subtractive quantization clustering processing module 610, receives the basic image 615 from the subtractive quantization clustering processing module 610, and outputs a basic code 635.
The lossy compression JPEG processing module 440 is connected to the difference processing module 620, receives the difference image 625 from the difference processing module 620, and outputs a difference code 645.

<< Fifth Embodiment >>
FIG. 7 is a conceptual module configuration diagram of a configuration example according to the fifth embodiment. In the fifth embodiment, a pre-quantization processing module 720 that pre-quantizes a flat block (a portion that does not correspond to a character edge) that does not include an edge is provided. In the pre-quantization processing module 720, quantization is performed according to the basis vector of the latter stage (lossy compression JPEG processing module 440). More specifically, quantization is performed with an average value (DC component) of 8 × 8 blocks. The total code amount of the basic image and the difference image is improved by aligning the quantization direction with JPEG for compressing the difference image.
The image processing apparatus 700 includes a flat block determination processing module 710, a pre-quantization processing module 720, a subtractive quantization clustering processing module 610, a difference processing module 620, a reversible compression flat (PNG) processing module 430, and an irreversible compression JPEG processing module 440. have.
The flat block determination processing module 710 and the pre-quantization processing module 720 quantize the flat portion of the target image 105.
The flat block determination processing module 710 is connected to the pre-quantization processing module 720 and the subtractive quantization clustering processing module 610, receives the target image 105, passes the flat block 714 to the pre-quantization processing module 720, and performs subtractive quantization. The non-flat block 712 is passed to the clustering processing module 610. The flat block determination processing module 710 extracts a flat portion in the target image 105 and passes it to the pre-quantization processing module 720 as a flat block 714. For example, the determination process of whether or not the surface is flat may be determined based on the D range or the number of colors of the block.

  The pre-quantization processing module 720 is connected to the flat block determination processing module 710 and the subtractive color quantization clustering processing module 610 and receives the flat block 714 from the flat block determination processing module 710. The pre-quantization processing module 720 performs an averaging process on, for example, an 8 × 8 pixel block for the flat block 714. By aligning the axes of JPEG and quantization, high frequency components are not left on the JPEG side. In this example, quantization is performed with a block average value (DC component) of 8 × 8 pixels so that the color reduction (rastering result) does not affect coefficients other than the DC component. Note that the quantization is not limited to 8 × 8 averaging (DC component), and quantization along the JPEG base direction is desirable in terms of the total code amount, but is not limited thereto.

The subtractive quantization clustering processing module 610 is connected to the flat block determination processing module 710, the pre-quantization processing module 720, the difference processing module 620, and the lossless compression flat (PNG) processing module 430. The non-flat block 712 is received and the basic image 615 is passed to the lossless compression Flate (PNG) processing module 430. The subtractive quantization clustering processing module 610 performs quantization for the image quantized by the pre-quantization processing module 720 to preserve the edges of the image. For example, a limited color process is performed.
The difference processing module 620 is connected to the subtractive color quantization clustering processing module 610 and the lossy compression JPEG processing module 440, receives the target image 105, and passes the difference image 625 to the lossy compression JPEG processing module 440.
The lossless compression Flate (PNG) processing module 430 is connected to the subtractive quantization clustering processing module 610, receives the basic image 615 from the subtractive quantization clustering processing module 610, and outputs a basic code 735.
The lossy compression JPEG processing module 440 is connected to the difference processing module 620, receives the difference image 625 from the difference processing module 620, and outputs a difference code 745.

<< Sixth Embodiment >>
Before describing the sixth embodiment, it will be described that the difference may underflow or overflow in the processing of the fourth embodiment.
FIG. 8 is an explanatory diagram illustrating an example in which the difference underflows or overflows. If the range of the target image 105 is 8 bits [0, 255], the difference value is [−255, 255]. Therefore, theoretically, the range may exceed the 8-bit range [−128, 127]. May be adversely affected. Of course, 8 bits is an example.
For example, if the pixel value 805 of the target image 105 is 250, and the pixel value 815 of the basic image 115 that is the processing of the edge preserving quantization module 110 is 10, the difference value that is the processing result of the difference processing module 120 Although 820 becomes 240, it becomes 127 of the range limit, and there are 125, 126, 127, etc. as irreversible results 840 that are processing results of the irreversible compression JPEG processing module 440.
Then, the decompression process for the irreversible result 840 is as follows.
・ 125 + 10 ≠ 250
・ 126 + 10 ≠ 250
127 + 10 ≠ 250
That is, the result does not return to the pixel value 250 of the original image.

FIG. 9 is a conceptual module configuration diagram of a configuration example according to the sixth embodiment. In the sixth embodiment, in order to prevent overflow and underflow, a difference value range control processing module 910 is provided to control the intensity of quantization and ensure image quality on the high image quality side. In the processing of the difference value range control processing module 910 of the sixth embodiment, control is performed so as not to be taken into a distant cluster. Specifically, a distance constraint is provided. For example, if the distance to the center of gravity of each cluster is not included in a cluster exceeding 127, the difference value can be guaranteed as [−127, 127]. In order to prevent being included in any cluster, for example, 128, which is an intermediate value of the pixel value range, may be added to the center of gravity.
The image processing apparatus 900 includes a difference value range control processing module 910, a subtractive color quantization clustering processing module 610, a difference processing module 620, a lossless compression flat (PNG) processing module 430, and a lossy compression JPEG processing module 440.
The difference value range control processing module 910 is connected to the subtractive color quantization clustering processing module 610 and receives the target image 105. The difference value range control processing module 910 controls the distance in cluster formation by the subtractive quantization clustering processing module 610 so that the processing result by the difference processing module 620 falls within a predetermined range. For example, a cluster as shown in FIG. 10 is generated. FIG. 10 is an explanatory diagram illustrating a processing example according to the sixth exemplary embodiment. An intermediate value is added to the center of gravity so that pixels not included in the cluster do not occur. Here, an example in which the color is divided into four colors (four clusters) and the distance constraint is 127 is shown.

The subtractive color quantization clustering processing module 610 is connected to the difference value range control processing module 910, the difference processing module 620, and the lossless compression Flate (PNG) processing module 430, receives the target image 105, and receives the lossless compression Flate (PNG) processing module. A basic image 615 is passed to 430.
The lossless compression Flate (PNG) processing module 430 is connected to the subtractive quantization clustering processing module 610, receives the basic image 615 from the subtractive quantization clustering processing module 610, and outputs a basic code 935.
The lossy compression JPEG processing module 440 is connected to the difference processing module 620, receives the difference image 625 from the difference processing module 620, and outputs a difference code 945.

FIG. 11 shows an example of a decompressed image of the basic image 1107B according to the sixth embodiment and background technology (JPEG-Progressive). The code amount when the basic image 1107B was compressed was controlled to be the same. Comparing the image 1110 of the extension result by the background art and the image 1120 of the extension result by the sixth embodiment, it can be seen that the image quality of the basic image is improved by the sixth embodiment.
FIG. 12 shows an example of the code amount of the basic image 1107B according to the sixth embodiment and background technology (JPEG-Progressive). The total code amount (the total of the basic image 1107B after compression processing and the difference image 1108B) can also be improved as compared with the background art (JPEG-Progressive).

FIG. 13 is an explanatory diagram showing a system configuration example using the present embodiment.
An example in which the processing of the present embodiment is provided as a service of the cloud distribution server 1300 is shown. An example in which a compressed image is transmitted from the cloud distribution server 1300 to each user terminal 1310 will be described. The communication line between the cloud distribution server 1300 and each user terminal 1310 may be wireless, wired, or a combination thereof, for example, the Internet or an intranet as a communication infrastructure. Here, it is assumed that the target image is generated as a code format structured such as low image quality (basic image) 1335, medium image quality 1345, and high image quality 1355.
As an example of distribution according to the line speed, a user terminal 1310A connected to a low-speed line transmits a code of only low image quality (basic image) 1335 and is connected to a medium-speed line. The low quality (basic image) 1335 and medium quality 1345 codes are transmitted to 1310B, and the low quality (basic image) 1335, medium quality 1345, and high quality 1355 are transmitted to the user terminal 1310C connected to the high-speed line. Is transmitted.
As an example of delivery in the image quality desired by the user, a code of only low image quality (basic image) 1335 is transmitted to the user terminal 1320A of the user who desires low image quality, and low image quality is transmitted to the user terminal 1320B of the user who desires medium image quality. Codes of (basic image) 1335 and medium image quality 1345 are transmitted, and codes of low image quality (basic image) 1335, medium image quality 1345 and image quality 1355 are transmitted to the user terminal 1320C who desires high image quality.
As an example of delivery in progressive display (an example of improving image quality over time), at time (t = 1), a code of only low image quality (basic image) 1335 is transmitted, and then (t = 2), Codes of low image quality (basic image) 1335 and medium image quality 1345 are transmitted, and then (t = 3), codes of low image quality (basic image) 1335, medium image quality 1345, and high image quality 1355 are transmitted. As a result, at the time (t = 1), the user terminal 1330 displays an image that is vague (“hazy” when compared to time (t = 2 or t = 3)), and the time (t = In 2), a clear image is displayed, and at time (t = 3), a clearer image is displayed.

<< Seventh Embodiment >>
In the above-described embodiments (the first to sixth embodiments), there are cases where the value range of the difference image does not fall within a predetermined value range. As described in the sixth embodiment, specifically, the value range of the difference image is [−255, 255]. Therefore, in the sixth embodiment, the pixel value of the basic image is corrected so that the pixel value of the difference image falls within [−128, 127]. Therefore, the image quality of the basic image becomes a problem. In addition, although it demonstrates using [-128,127] as a "predetermined value range", as above-mentioned, this is an illustration.
In the seventh embodiment, as in the previous embodiment, a basic image in which gradation is quantized while leaving a character edge of the image, and a difference image that is expressed by a difference between pixel values of the original image and the basic image are generated. To do. If the value range of the difference image does not fit in [−128, 127], the difference image is divided into a texture image and an offset image.

Terms used in the following description will be described with specific examples.
The basic image is an image in which the edge of the target image is stored, and the value range is [0, 255].
The difference image is an image obtained by subtracting the basic image from the target image, and the range is [−256, 255].
The offset value is a value for offsetting the difference image so as to fall within the range of [−128, 127], and the value range is [−127, 128].
An offset image is an image whose pixel value is composed of an offset value, and its value range is [−127, 128].
A texture image is an image obtained by subtracting an offset image from a difference image, and its value range is [−128, 127].

FIG. 14 is a conceptual module configuration diagram of a configuration example according to the seventh embodiment.
The image processing apparatus 1400 includes an edge preserving quantization module 110, a difference processing module 120, an image division module 1422, a compression A processing module 130, and a compression B processing module 140. An image processing apparatus 1400 is obtained by adding an image division module 1422 to the first embodiment.
The difference processing module 120 is connected to the edge preserving quantization module 110 and the image division module 1422, receives the target image 105, and passes the difference image 125 to the image division module 1422.

  The image segmentation module 1422 is connected to the difference processing module 120 and the compression B processing module 140, receives the difference image 125 from the difference processing module 120, and passes the difference image / texture image and offset image 1425 to the compression B processing module 140. . When the image generated by the difference processing module 120 does not fall within a predetermined range, the image division module 1422 is a first that fits the image (the difference image 125 generated by the difference processing module 120) within that range. Divide into an image and a second image. Specifically, when the range of the difference image 125 does not fit in [−128, 127], the difference image 125 is divided into a texture image and an offset image. That is, the difference image is divided into two images that can be expressed in 8 bits. By this processing, deterioration of the image quality of the basic image 115 is avoided.

  The compression B processing module 140 is connected to the image division module 1422, receives the difference image / texture image and the offset image 1425 from the image division module 1422, and outputs the difference code 145. The compression B processing module 140 compresses the first image and the second image divided by the image division module 1422. Specifically, the texture image and the offset image that can be expressed in 8 bits are each compressed. Although there are two images to be compressed, the compression method itself may be the compression method in the above-described embodiment. Therefore, the difference code 145 includes two image codes (texture image and offset image compression results).

The image processing apparatus 1450 includes an expansion A processing module 160, an expansion B processing module 170, and an addition processing module 180.
The decompression B processing module 170 is connected to the addition processing module 180, receives the difference code 145, and passes the difference image / texture image and offset image 1475 to the addition processing module 180. The decompression B processing module 170 decompresses the first code and the second code generated by the compression B processing module 140 of the image processing apparatus 1400. Specifically, the compression results of the texture image and the offset image are expanded. Therefore, two images (texture image and offset image) are decoded as the difference image.

  The addition processing module 180 is connected to the decompression A processing module 160 and the decompression B processing module 170, receives the basic image 165 from the decompression A processing module 160, and receives the difference image / texture image and offset image 1475 from the decompression B processing module 170. And an expanded image 195 is output. The addition processing module 180 adds the first image and the second image expanded by the expansion B processing module 170 to the image expanded by the expansion A processing module 160. Specifically, three images ((1) basic image 165 decoded by the decompression A processing module 160, (2) difference image / texture image decoded by the decompression B processing module 170, and texture images in the offset image 1475) (3) The difference image / texture image decoded by the decompression B processing module 170 and the offset image in the offset image 1475 are added to generate the decompressed image 195.

FIG. 15 is a flowchart illustrating a processing example according to the seventh exemplary embodiment (image processing apparatus 1400). An example of processing when compression is performed and the range of the difference image is [−128, 127] is shown.
In step S1502, the edge preserving quantization module 110 extracts edges from the target image.
In step S1504, the edge preserving quantization module 110 performs gradation quantization.
In step S1506, the edge preserving quantization module 110 generates a basic image I.
In step S1508, the difference processing module 120 generates a difference image J.
In step S1510, the compression A processing module 130 compresses the basic image I by compression A.
In step S <b> 1512, the compression B processing module 140 compresses the difference image J by the compression B.

FIG. 16 is a flowchart illustrating a processing example according to the seventh exemplary embodiment (image processing apparatus 1400). An example of processing when compression is performed and the range of the difference image does not fit in [−128, 127] is shown.
In step S1602, the edge preserving quantization module 110 extracts edges from the target image.
In step S1604, the edge preserving quantization module 110 performs gradation quantization.
In step S1606, the edge preserving quantization module 110 generates a basic image I.
In step S1608, the difference processing module 120 generates a difference image J.
In step S1610, the image division module 1422 divides the difference image J. The processing in step S1610 will be described later using the flowchart shown in the example of FIG.
In step S1612, the compression A processing module 130 compresses the basic image I by the compression A.
In step S <b> 1614, the compression B processing module 140 compresses the difference image J by the compression B.
Note that it may be determined using the range of the difference image J which of the flowchart shown in the example of FIG. 15 and the flowchart shown in the example of FIG. Specifically, when the range of the difference image J is within a predetermined range, the process according to the flowchart shown in the example of FIG. 15 is performed, and the range of the difference image J does not fall within the predetermined range. The process according to the flowchart shown in the example of FIG. 16 is performed.

FIG. 17 is a flowchart illustrating a processing example according to the seventh exemplary embodiment (image processing apparatus 1450). An example of processing in the case of decompression processing when the value range of the difference image is [−128, 127] is shown.
In step S1702, the decompression A processing module 160 decompresses the basic image by decompression A.
In step S1704, the decompression B processing module 170 decompresses the differential image by the decompression B.
In step S1706, the addition processing module 180 adds the basic image and the difference image.

FIG. 18 is a flowchart illustrating a processing example according to the seventh exemplary embodiment (image processing apparatus 1450). An example of processing in a case where the range of the difference image does not fit in [−128, 127] is shown.
In step S1802, the decompression A processing module 160 decompresses the basic image by decompression A.
In step S1804, the decompression B processing module 170 decompresses the texture image by the decompression B.
In step S1806, the addition processing module 180 performs image addition. That is, it is the addition of the processing result in step S1802 and the processing result in step S1804.
In step S1808, the decompression B processing module 170 decompresses the offset image by decompression B.
In step S1810, the addition processing module 180 performs image addition. That is, it is the addition of the processing result in step S1806 and the processing result in step S1808.
It should be noted that whether the flowchart shown in the example of FIG. 17 or the flowchart shown in the example of FIG. 18 is used depends on whether the decompression B processing module 170 receives one code (encoded image) or two. You may make it judge by whether there exists. Specifically, when the decompression B processing module 170 has received one code, the process shown in the flowchart of FIG. 17 is performed, and when the decompression B processing module 170 has received two codes. Then, the processing according to the flowchart shown in the example of FIG. 18 is performed. In either case, the processing according to the flowchart shown in the example of FIG. 18 may be performed. This is because if there is no second code, the processing in steps S1808 and S1810 is substantially equivalent to nothing.

FIG. 19 is a flowchart illustrating a processing example (processing example of step S1610) according to the seventh embodiment.
In step S1902, the minimum value f and the maximum value c of the difference image are calculated.
In step S1904, it is determined whether or not f <−128. If f <−128, the process proceeds to step S1906. Otherwise, the process proceeds to step S1908.
In step S1906, the offset value s is calculated by the following equation. s = f + 128
In step S1908, it is determined whether or not c> 127. If c> 127, the process proceeds to step S1910. Otherwise, the process ends (step S1999). That is, it is determined whether or not the value range of the difference image is within a predetermined value range in the determination processing in steps S1904 and S1908.
In step S1910, the offset value s is calculated by the following equation. s = c-127
In step S1912, an offset image S in which all pixel values are offset values s is generated.
In step S1914, the offset image S is subtracted from the difference image J to generate a texture image T.

  For the sake of brevity, the seventh embodiment will be described below using an extreme example. In the seventh embodiment, it is assumed that the compression / decompression A is PNG and the compression / decompression B is JPG. Here, methods other than the above may be used for the compression / decompression A and B, and the same method may be used for the compression / decompression A and the compression / decompression B. In the seventh embodiment, a color image in the RGB color space is used as a target image. However, the present technology can be applied to all color images and grayscale images.

FIG. 20 is an explanatory diagram illustrating a processing example according to the seventh exemplary embodiment. Assume that the image 2000 shown in the example of FIG. 20A is given as the target image. Here, in FIG. 20, the value range of the target image and the basic image will be described as [0, 255], and the value range of the difference image will be described as [−255, 255]. The edge extraction / quantization processing module 110 obtains a basic image 2040 (FIG. 20B) and a difference image 2050 (FIG. 20C) as a difference between the image 2000 and the basic image 2040. The image 2000 is divided into a region 2010 (6 × 6 pixel square), a region 2020 (inverted L shape with a width of 1 pixel), and a region 2030 (inverted L shape with a width of 1 pixel), each of which is configured with the same color. In the example, the difference image 2050 is also divided into a region 2060 (6 × 6 pixel square), a region 2070 (inverted L shape with a width of 1 pixel), and a region 2080 (inverted L shape with a width of 1 pixel). It will be.
At this time, the range of the difference image 2050 does not fall within [−128, 127]. In the sixth embodiment, in order to guarantee the range, as shown in the example of FIG. 21A, an intermediate value (128) is added to the basic image 2100 so that the range of the difference image falls within [−128, 127]. )use. However, the image quality of the basic image 2100 deteriorates by replacing with the intermediate value.

差分画像の画素値の最大値をｃ、最小値をｆとすると、式８が成立する。

また、

が成立する。 Here, when the pixel value of the target image is X _{k, l} , the pixel value of the basic image is I _{k, l} , and the pixel value of the difference image is J _{k, l} , Expressions 4 to 7 are established. k and l represent indexes on the image.

If the maximum value of the pixel values of the difference image is c and the minimum value is f, Expression 8 is established.

Also,

Is established.

式１１が成立し、式７〜１１より、式１２が成立する。

オフセット画像をＳ_ｋ，ｌ、テクスチャ画像をＴ_ｋ，ｌとし、式１３及び式１４で定義する。

式１１〜１４より、オフセット画像とテクスチャ画像の取り得る値は式１５、式１６となる。

以上より、差分画像Ｊ_ｋ，ｌはオフセット画像Ｓ_ｋ，ｌとテクスチャ画像Ｔ_ｋ，ｌで表すことができる。 If the offset value s is defined by Equation 10,

Expression 11 is established, and Expression 12 is established from Expressions 7 to 11.

The offset image is S _{k, l} and the texture image is T _{k, l} , which are defined by Expression 13 and Expression 14.

From Equations 11-14, the values that the offset image and texture image can take are Equations 15 and 16.

As described above, the difference image J _{k, l} can be represented by the offset image S _{k, l} and the texture image T _{k, l} .

  In the case of the example shown in FIG. 20, when the process according to the seventh embodiment is performed, a basic image and a difference image as shown in the example of FIG. 22 are obtained. Specifically, the basic image 2200 is the same as the basic image 2040 shown in the example of FIG. The difference image 2050 shown in the example of FIG. 20C is divided into a difference image A 2210 shown in the example of FIG. 22B and a difference image B 2250 shown in the example of FIG. The difference image B2250 is composed of pixels having the same value and is an offset image, and its value range is [−127, 128]. Then, as a difference image between the difference image 2050 and the difference image B2250 shown in the example of FIG. 20C, a difference image A2210 shown in the example of FIG. 22B is generated. The difference image A 2210 is divided into a region 2220 (6 × 6 pixel square), a region 2230 (inverted L shape with a width of 1 pixel), and a region 2240 (inverted L shape with a width of 1 pixel). The range is [−128, 127].

Next, the transition of the image in the decompression (decoding) process is shown in the example of FIG. From the example of FIG. 23, it can be confirmed that the image quality of the basic image does not deteriorate according to the seventh embodiment.
The (basic image + difference image) 2310 includes a region 2320 (6 × 6 pixel square), a region 2330 (inverted L shape with a width of 1 pixel), and a region 2340 (inverted L with a width of 1 pixel) each configured with the same color. Form).
In the sixth embodiment, the basic image 2300 shown in the example of FIG. 23A1 (equivalent to the basic image 2100 shown in the example of FIG. 21A) is replaced with an intermediate value. The image quality is degraded. Therefore, the image quality of the final expanded image (basic image + difference image) 2310 also deteriorates.
On the other hand, in the seventh embodiment, the basic image 2350 shown in the example of FIG. 23B1 (equivalent to the basic image 2040 shown in the example of FIG. 20B) is used as it is, and a texture image is used as the basic image 2350. A difference image A2210 is added to generate (basic image + texture image) 2360 shown in the example of FIG. 23B2, and further, a difference image B2250 that is an offset image is added to (basic image + texture image) 2360. Then, (basic image + texture image + offset image) 2380 shown in the example of FIG. 23 (B3) is generated.
Here, since the value range of the offset image is [−127, 128] (see Expression 15), it is necessary to control the behavior with the offset image and the texture image in the decoding process. However, this reduces the versatility of the seventh embodiment. Therefore, for example, 1 is subtracted from the pixel value 128 of the offset image to set the value range to [−127, 127]. In this case, degradation of the image quality with a pixel value of 1 occurs at maximum in the decoded image, but when irreversible compression technology such as JPG is used for the difference image, the image quality is degraded by the irreversible compression technology. Image quality degradation due to the method is negligible.

  Next, the code amount will be described. The code amount of the texture image of the seventh embodiment corresponds to the code amount of the difference image in the sixth embodiment. Regarding the code amount of the basic image, in the sixth embodiment, since the discontinuity of the pixel value occurs by using the intermediate value for the basic image, the code amount of the basic image of the sixth embodiment is the seventh. This is larger than the code amount of the basic image in the embodiment. In the seventh embodiment, information corresponding to the increase in the code amount is given to the offset image. Therefore, the code amount of the offset image in the seventh embodiment corresponds to an increase in the code amount of the basic image in the sixth embodiment. From the above, it can be seen that the total code amount of the seventh embodiment is substantially equal to the total code amount of the sixth embodiment.

  A hardware configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 24 is configured by a personal computer (PC), for example, and shows a hardware configuration example including a data reading unit 2417 such as a scanner and a data output unit 2418 such as a printer.

  A CPU (Central Processing Unit) 2401 is the various modules described in the above-described embodiments, that is, the edge preserving quantization module 110, the difference processing module 120, the compression A processing module 130, the compression B processing module 140, and the decompression A processing. Module 160, decompression B processing module 170, addition processing module 180, lossless compression Flate (PNG) processing module 430, lossy compression JPEG processing module 440, lossless decompression Deflate (PNG) processing module 460, lossy decompression JPEG processing module 470, Binarization processing module 510, difference processing module 520, subtractive color quantization clustering processing module 610, difference processing module 620, flat block determination processing module 7 10, a control unit that executes processing according to a computer program that describes an execution sequence of each module such as a pre-quantization processing module 720 and a difference value range control processing module 910.

  A ROM (Read Only Memory) 2402 stores programs used by the CPU 2401, operation parameters, and the like. A RAM (Random Access Memory) 2403 stores programs used in the execution of the CPU 2401, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 2404 including a CPU bus or the like.

  The host bus 2404 is connected to an external bus 2406 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 2405.

  A keyboard 2408 and a pointing device 2409 such as a mouse are input devices operated by an operator. The display 2410 includes a liquid crystal display device, a CRT (Cathode Ray Tube), and the like, and displays various types of information as text and image information.

  An HDD (Hard Disk Drive) 2411 includes a hard disk (may be a flash memory or the like), drives the hard disk, and records or reproduces a program executed by the CPU 2401 and information. The hard disk stores the target image 105, the basic code 135, the differential code 145, the basic image 115, the differential image 125, the basic image 165, the differential image 175, and the like. Further, various other data, various computer programs, and the like are stored.

  The drive 2412 reads data or a program recorded on a removable recording medium 2413 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and reads the data or program into an interface 2407 and an external bus 2406. , The bridge 2405, and the RAM 2403 connected via the host bus 2404. The removable recording medium 2413 can also be used as a data recording area similar to the hard disk.

  The connection port 2414 is a port for connecting the external connection device 2415 and has a connection unit such as USB, IEEE1394. The connection port 2414 is connected to the CPU 2401 and the like via the interface 2407, the external bus 2406, the bridge 2405, the host bus 2404, and the like. A communication unit 2416 is connected to a communication line and executes data communication processing with the outside. The data reading unit 2417 is, for example, a scanner, and executes document reading processing. The data output unit 2418 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus illustrated in FIG. 24 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 24, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 24 may be connected to each other via communication lines so as to cooperate with each other. In particular, in addition to personal computers, portable information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc.), information appliances, copiers, fax machines, scanners, printers, multifunction devices (scanners, printers, copiers) Or an image processing apparatus having two or more functions such as a fax machine).

  Note that the above-described various embodiments may be combined (for example, adding or replacing a module in one embodiment in another embodiment), and processing contents of each module The technique described in the background art may be employed.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

You may grasp | ascertain the above-mentioned embodiment (especially 1st Embodiment-6th Embodiment) as follows. These may be combined with the invention according to the seventh embodiment.
[A] quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
First compression means for compressing the quantized image quantized by the quantization means;
An image processing apparatus comprising: a second compression unit that compresses the difference image generated by the difference image generation unit.
[B] first decompression means for decompressing the code generated by the first compression means of the image processing apparatus of [A];
Second decompression means for decompressing the code generated by the second compression means of the image processing apparatus of [A];
An image processing apparatus comprising: an adding unit that adds the image expanded by the second expansion unit to the image expanded by the first expansion unit.
[C] quantization means for performing quantization while preserving the edge of the image;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
First compression means for compressing the quantized image quantized by the quantization means;
Second compression means for compressing the difference image generated by the difference image generation means;
First decompression means for decompressing the code generated by the first compression means;
Second decompression means for decompressing the code generated by the second compression means;
An image processing apparatus comprising: an adding unit that adds the image expanded by the second expansion unit to the image expanded by the first expansion unit.
[D] The first compression means performs a reversible compression process,
The image processing apparatus according to [A] or [C], wherein the second compression unit performs lossy compression processing.
[E] The image processing apparatus according to any one of [A], [C], and [D], wherein the quantization unit performs binarization processing as quantization.
[F] The image processing apparatus according to any one of [A], [C], and [D], wherein the quantization unit performs a limited color processing as quantization.
[G] further comprising second quantization means for quantizing a flat portion of the image,
[A], [C], wherein the quantization means performs quantization for preserving edges of the second quantized image quantized by the second quantization means. , [D], [E], and [F].
[H] It further comprises control means for controlling a distance in cluster formation by the quantization means for performing the limited color processing so that the processing result by the difference image generation means falls within a predetermined range. The image processing apparatus according to [F].
[I] Computer
A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
First compression means for compressing the quantized image quantized by the quantization means;
The image processing program for functioning as a 2nd compression means to compress the difference image produced | generated by the said difference image production | generation means.
[J] first decompression means for decompressing the code generated by the first compression means of the image processing apparatus of [A];
Second decompression means for decompressing the code generated by the second compression means of the image processing apparatus of [A];
An image processing apparatus comprising: a display unit that displays the image expanded by the first expansion unit first with respect to the image added by the addition unit.
[K]
First decompression means for decompressing the code generated by the first compression means of the image processing program of [I];
Second decompression means for decompressing the code generated by the second compression means of the image processing program of [I],
An image processing program for causing an image decompressed by the first decompressing means to function as a display means for displaying the image added by the adding means first.

According to the image processing apparatus [A], when compressing an image, it is possible to improve the image quality of the first decompressed image as compared to Progressive-JPEG.
According to the image processing apparatus [B], in the case of decompressing a compressed image, the image quality of the first image can be improved as compared with Progressive-JPEG.
According to the image processing apparatus [C], in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.
According to the image processing apparatus of [D], the code amount can be reduced as compared with the case where the second compression unit performs the lossless compression process.
According to the image processing apparatus of [E], in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.
According to the image processing apparatus [F], in the case of compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.
According to the image processing apparatus [G], it is possible to reduce the code amount as compared with the case where the original image is quantized.
According to the image processing apparatus [H], the image quality can be improved as compared with the case where the difference underflows or overflows.
According to the image processing program [I], when compressing an image, the image quality of the image after the first decompression can be improved as compared with Progressive-JPEG.
According to the image processing apparatus [J], it is possible to improve the image quality of an image to be displayed earlier as compared to Progressive-JPEG.
According to the image processing program [K], it is possible to improve the image quality of an image to be displayed earlier as compared with Progressive-JPEG.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 105 ... Object image 110 ... Edge preservation | save quantization module 115 ... Basic image 120 ... Difference processing module 125 ... Difference image 130 ... Compression A processing module 135 ... Basic code 140 ... Compression B processing module 145 ... Difference code 150 Image processing device 160 ... Decompression A processing module 165 ... Basic image 170 ... Decompression B processing module 175 ... Difference image 180 ... Addition processing module 195 ... Decompression image 400 ... Image processing device 430 ... Reversible compression Flate (PNG) processing module 435 ... Basic code 440 ... lossy compression JPEG processing module 445 ... difference code 450 ... image processing device 460 ... lossless decompression deflate (PNG) processing module 470 ... lossy decompression JPEG processing module 495 ... decompressed image 500 ... image Processing device 510 ... Binarization processing module 515 ... Basic image 520 ... Difference processing module 525 ... Difference image 535 ... Basic code 545 ... Difference code 600 ... Image processing device 610 ... Subtractive color quantization clustering processing module 615 ... Basic image 620 ... Difference Processing module 625 ... difference image 635 ... basic code 645 ... difference code 700 ... image processing device 710 ... flat block determination processing module 712 ... non-flat block 714 ... flat block 720 ... pre-quantization processing module 735 ... basic code 745 ... difference code 900 ... Image processing device 910 ... Difference value range control processing module 935 ... Basic code 945 ... Difference code 1300 ... Cloud distribution server 1310 ... User terminal 1320 ... User terminal 1330 ... User terminal 1400 ... Image processing device 1422 ... image segmentation module 1425 ... difference image / texture image, the offset image 1450 ... image processing apparatus 1475 ... difference image / texture image, offset image

Claims (13)

A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
A dividing unit that divides the image into a first image and a second image that fall within the range when the image generated by the difference image generation unit does not fall within a predetermined range;
First compression means for compressing the quantized image quantized by the quantization means;
An image processing apparatus comprising: a second compression unit that compresses the first image and the second image divided by the division unit. First decompression means for decompressing the code generated by the first compression means of the image processing apparatus according to claim 1;
A second decompression unit for decompressing the first code and the second code generated by the second compression unit of the image processing apparatus according to claim 1;
An image processing apparatus comprising: an adding unit that adds the first image and the second image expanded by the second expansion unit to the image expanded by the first expansion unit. A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
A dividing unit that divides the image into a first image and a second image that fall within the range when the image generated by the difference image generation unit does not fall within a predetermined range;
First compression means for compressing the quantized image quantized by the quantization means;
Second compression means for compressing the first image and the second image divided by the dividing means;
First decompression means for decompressing the code generated by the first compression means;
Second decompression means for decompressing the first code and the second code generated by the second compression means;
An image processing apparatus comprising: an adding unit that adds the first image and the second image expanded by the second expansion unit to the image expanded by the first expansion unit. The first compression means performs a reversible compression process,
The image processing apparatus according to claim 1, wherein the second compression unit performs irreversible compression processing. The image processing apparatus according to claim 1, wherein the quantization unit performs binarization processing as quantization. The image processing apparatus according to claim 1, wherein the quantization unit performs limited color processing as quantization. Second quantization means for quantizing a flat portion of the image,
The said quantization means performs the quantization which preserve | saves the edge of this image for the 2nd quantized image quantized by the said 2nd quantization means. The image processing apparatus according to any one of 5 and 6. The apparatus further comprises a control unit that controls a distance in cluster formation by the quantization unit that performs the limited color processing so that a processing result by the difference image generation unit falls within a predetermined range. Item 7. The image processing apparatus according to Item 6. Computer
A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
A dividing unit that divides the image into a first image and a second image that fall within the range when the image generated by the difference image generation unit does not fall within a predetermined range;
First compression means for compressing the quantized image quantized by the quantization means;
An image processing program for functioning as a second compression unit that compresses the first image and the second image divided by the dividing unit. First decompression means for decompressing the code generated by the first compression means of the image processing apparatus according to claim 1;
A second decompression unit for decompressing the code generated by the second compression unit of the image processing apparatus according to claim 1;
An image processing apparatus comprising: a display unit that displays the image expanded by the first expansion unit first with respect to the image added by the addition unit. Computer
First decompression means for decompressing the code generated by the first compression means of the image processing program of claim 9;
A second decompression means for decompressing the first code and the second code generated by the second compression means of the image processing program according to claim 9;
An image processing program for causing an image expanded by the first expansion unit to function as an adding unit that adds the first image and the second image expanded by the second expansion unit. Computer
A quantization means for performing quantization while preserving image edges;
Difference image generation means for generating a difference image between the image and the quantized image quantized by the quantization means;
A dividing unit that divides the image into a first image and a second image that fall within the range when the image generated by the difference image generation unit does not fall within a predetermined range;
First compression means for compressing the quantized image quantized by the quantization means;
Second compression means for compressing the first image and the second image divided by the dividing means;
First decompression means for decompressing the code generated by the first compression means;
Second decompression means for decompressing the first code and the second code generated by the second compression means;
An image processing program for causing an image expanded by the first expansion unit to function as an adding unit that adds the first image and the second image expanded by the second expansion unit. Computer
First decompression means for decompressing the code generated by the first compression means of the image processing program of claim 9;
A second decompression means for decompressing the code generated by the second compression means of the image processing program according to claim 9;
An image processing program for causing an image decompressed by the first decompressing means to function as a display means for displaying the image added by the adding means first.

Images