JPH0884257A - Image processing unit and its method - Google Patents

Abstract

PURPOSE: To obtain an image processing unit and its method for improving the coding efficiency by eliminating noise in a color image so as to make an edge clear. CONSTITUTION: The user designates the number of colors to be extracted as 17 representative color by a representative color number designation device 4, and a representative color extract device 5 extracts the number of representative colors designated from image data stored in a frame memory 2 and a background color extract device 3 extracts a background color from the image data respectively. The extracted representative color and background color are coded by a color coder 10 and the image data stored in the frame memory 2 are converted by a color converter 6 and coded based on the representative color and background color.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method thereof, and more particularly to an image processing apparatus and method for encoding and decoding color images.

[0002]

2. Description of the Related Art In recent years, there are increasing opportunities to store and transmit coded color images. As a color image encoding method, the JPEG method has been standardized and is widely used in general. The JPEG method divides a natural image into blocks and performs discrete cosine transform (hereinafter referred to as "DCT").
This is a method of quantizing and coding the coefficient. Details are introduced in "International Standards for Multimedia Coding" (edited by Hiroshi Yasuda and published by Maruzen Publishing Co., Ltd.), so explanations are omitted.

FIG. 1 is a block diagram showing a configuration of an encoding unit of an image processing apparatus which performs JPEG encoding.

In FIG. 1, an image input device 1001 is a scanner for inputting color image data, a storage device for storing image data, or the like. 1002 is a frame memory that stores color image data for one screen. Reference numeral 1003 is a DCT device that performs DCT, and 1004 is a quantizer that quantizes DCT coefficients.
An encoder 1005 assigns a code to the quantization result or generates various identification codes. Also, 1006
Is an output device that outputs or stores the generated code data to the communication circuit.

Image data input from the image input device 1001 is stored in a frame memory 1002, divided into blocks of a predetermined size, and input to a DCT device 1003. DCT device
The DCT coefficient output from 1003 is input to the quantizer 1004 and quantized with a predetermined quantization width. And the encoder
A code is assigned at 1005, and code data is generated.
The generated code data is sent to the output device 1006.

FIG. 2 is a block diagram showing the structure of the decoding unit of the image processing apparatus.

In the figure, reference numeral 1011 denotes an input device for inputting code data from a communication circuit or a storage device. Decoder 1012 decodes various identification codes and code data and outputs quantized data. Reference numeral 1013 is an inverse quantizer that reproduces DCT coefficients by inverse quantization, and 1014 is an IDCT device that performs inverse DCT. A frame memory 1015 stores reproduced image data for one screen. An image output device 1016 is a printer or a storage device that outputs image data.

The coded data input from the input device 1011 is decoded by the decoder 1012, inversely quantized by the inverse quantizer 1013 with a predetermined quantization width, and the DCT coefficient is reproduced. The reproduced coefficient is converted by the IDCT device 1014, and the image data is reproduced and stored in the frame memory 1015. The reproduced image data is printed out or stored by the image output device 1016.

[0009]

However, the above-mentioned conventional example has the following problems.

In a color image coding system such as JPEG, the high frequency components are impaired, the edges of the image are blunted, and the image quality tends to be greatly deteriorated. Specifically, a peculiar folding distortion (generally called mosquito noise) occurs around the edge of the image. This is a problem peculiar to transform coding in which the quantization width is increased for high-frequency components of transform coefficients in order to improve coding efficiency. Therefore,
Mosquito noise is superimposed on a portion that should have a uniform density (or brightness), resulting in deterioration of image quality that impairs uniformity of density (or brightness).

The present invention is intended to solve the above-mentioned problem, and eliminates noise in a color image by utilizing the fact that a color original is often expressed by a limited number of colors. An object of the present invention is to provide an image processing device and method for improving coding efficiency by clarifying edges.

[0012]

[Means for Solving the Problems] and

The present invention has the following structure as one means for achieving the above object.

An image processing apparatus according to the present invention is an image processing apparatus which encodes input image data, and comprises designation means for designating the number of colors and representative colors of the number designated by the designation means. Extracting means for extracting from the image data, first encoding means for encoding information of the representative color extracted by the extracting means, and converting means for converting the image data based on the representative color, A second encoding means for encoding the image data converted by the converting means.

Further, in the image processing apparatus for encoding the input image data, the extracting means for extracting an arbitrary number of representative colors from the image data, and the representative color extracted by the extracting means. A first encoding means for encoding information, a converting means for converting the image data based on the representative color, and a second encoding means for encoding the image data converted by the converting means. It is characterized by having.

An image processing apparatus for decoding encoded image data, comprising: a first decoding means for decoding information of a representative color included in the encoded data; and image data included in the encoded data. It is characterized by having a second decoding means for decoding and a conversion means for converting the image data decoded by the second decoding means based on the representative color decoded by the first decoding means.

An image processing method according to the present invention is an image processing method for encoding image data, wherein a designation step for designating the number of colors and a representative number of representative colors designated in the designation step are applied to the image data. An extraction step to extract from
A first encoding step of encoding the information of the representative color extracted in the extracting step, a converting step of converting the image data based on the representative color, and encoding the image data converted in the converting step. And a second encoding step.

An image processing method for encoding image data, comprising an extracting step of extracting an arbitrary number of representative colors from the image data, and encoding the information of the representative colors extracted in the extracting step. A first encoding step to
It is characterized by including a conversion step of converting the image data based on the representative color and a second encoding step of encoding the image data converted in the conversion step.

Further, in the image processing method for decoding the coded image data, the first decoding step of decoding the information of the representative color included in the code data, and the image data included in the code data are performed. It is characterized by including a second decoding step of decoding and a conversion step of converting the image data decoded in the second decoding step based on the representative color decoded in the first decoding step.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image processing apparatus according to an embodiment of the present invention will be described in detail below with reference to the drawings.

[0020]

[First Embodiment] FIG. 3 is a block diagram showing a configuration example of an encoding unit of an image processing apparatus according to the present invention.

In FIG. 1, reference numeral 1 denotes an image input device, such as a scanner for inputting color image data and a storage device for accumulating image data. A frame memory 2 stores color image data for one screen. A background color extractor 3 extracts the background color of the input image data. A representative color number designator 4 is for designating the number of representative colors representing the input image data. A representative color extractor 5 extracts a representative color from the input image data. The 6-color replacer replaces the image data input with the extracted background color and representative color. Reference numeral 7 is a DCT device that performs DCT, and 8 is a quantizer that quantizes DCT coefficients. The encoder 9 assigns a code to the quantization result and generates various identification codes. A color encoder 10 encodes the background color and the representative color. An output device 11 outputs or stores the generated code data to the communication circuit.

The image data input from the image input device 1 is stored in the frame memory 2 and then input to the background color extractor 3. The background color extractor 3 calculates the maximum frequency image data value from the histogram of the input image data. Then, the calculated image data value is used as the background color. The user designates the number of colors required to represent the input image using the representative color number designator 4.

The representative color extractor 5 inputs the background color from the background color extractor 3 and the representative color number from the representative color number designating unit 4, respectively, and then inputs the image data from the frame memory 2. The input background color and image data are converted to CIE1976 L * a * b * color space data. Then, the color difference between the input background color and the image data is obtained, compared with a preset threshold Th1, and a histogram is created for image data whose color difference is larger than the threshold Th1. This histogram is used to select a representative color. First, the color with the highest appearance frequency is selected as the first representative color. Subsequently, the representative color extractor 5 sets the frequency of the colors whose color difference between the selected representative color and the representative color is equal to or less than a preset threshold Th2 to zero. Hereinafter, the color having the highest appearance frequency is selected as the representative color, and the operation of setting the frequency of the colors in the vicinity to zero is repeated to select the designated number of representative colors.

After the representative color is extracted in this way, the background color is first input from the background color extractor 3 to the color encoder 10. The color encoder 10 first generates a representative color start code indicating the start of the representative color code data and outputs the representative color start code to the output device 11, and then the L * a * b * color space data of the background color to the output device 11. Output. Subsequently, the number of representative colors is input from the representative color number designator 4, the L * a * b * color space data of each representative color is input from the representative color extractor 5, and these are output to the output device 11. After that, a JPEG start code indicating the start of the JPEG code data is generated and output to the output device 11. A unique code that does not occur in JPEG coding is used as the start code.

Then, the encoder 9 outputs the header data defined by JPEG to the output device 11, and the encoding of the image data is started. The color converter 6 first sets the background color and each representative color L
The * a * b * color space data is input, and then the pixels forming a block of a predetermined size are sequentially read from the frame memory 2. After converting the input pixel data to L * a * b * color space data,
The color difference from the background color or each representative color is obtained, and the background color or the representative color with the smallest color difference is substituted. Then, when the replacement of all pixel data of the block is completed, the data of the block is output to the DCT unit 7.

The DCT unit 7 performs DCT on the input block, the quantizer 8 quantizes the DCT coefficient with a predetermined quantization width, and the encoder 9 assigns a code to the quantized result. Then, the output device 11 outputs or stores the generated code data to the communication line. In this way, when all the image data stored in the frame memory 2 has been encoded, the encoding ends.

FIG. 4 is a block diagram showing a configuration example of the decoding unit of the image processing apparatus according to the present invention.

In the figure, reference numeral 21 is an input device for inputting code data from a communication circuit or a storage device. A start code detector 22 analyzes the input start code. A color decoder 23 decodes the number of background colors, representative colors, and L * a * b * image data. A decoder 24 decodes the identification code and the coded data and outputs the quantized data. 25 is an inverse quantizer which reproduces DCT coefficients by inverse quantization, and 26 is inverse DCT
It is an IDCT device. A color converter 28 converts the image data input via the selector 27 into a decoded background color or representative color. Reference numeral 30 denotes a frame memory, which stores the reproduced image data input via the selector 29 for one screen. An image output device 31 is a printer or a storage device that outputs image data.

The code data input from the input device 21 is
It is input to the start code detector 22 and the star and the code are detected. The start code detector 22 sets the coding mode flag to the representative color mode when the first start code is the representative color start code. If the first start code is the JPEG start code, the coding mode flag is set to the JPEG mode. This coding mode flag is input to the selectors 27 and 29.

First, the case where the encoding mode is the representative color mode will be described. Then start code detector 22
The code data input to is sent to the color decoder 23. Color decoder 23, after decoding the L * a * b * color space data of the background color,
The number of representative colors is decoded, and the L * a * b * color space data (representative color) for that number is decoded. Each decoded data is a color converter
Input to 28 and set. When the start code detector 22 detects a JPEG start code, the start code detector 22 sends the code data to be input subsequently to the decoder 24. Then, the decoded quantized coefficient is input to the inverse quantizer 25 and inversely quantized with a predetermined quantization width. The dequantized DCT coefficient is the IDCT unit 2
Input to 6 and image data is reproduced. At this time, since the representative color mode is set in the selectors 27 and 29, the reproduced image data is processed by the color converter 28.

The color converter 28 inputs the pixel data forming the block sequentially output from the IDCT device 26, converts the pixel data into L * a * b * color space data, and then the data, the background color and each representative. Find the color difference from the color. Then, after replacing the pixel data with the background color or the representative color having the smallest color difference, the pixel data is converted into the input data format (for example, RGB) and output. The image data output from the color converter 28 is the selector.
The image output device is stored in the frame memory 30 via 29.
31 prints out or stores this data.

Next, the case where the encoding mode is the JPEG mode will be described. Subsequently, the code data input to the start code detector 22 is sent to the decoder 24. And
The image data is reproduced as in the representative color mode. At this time, since the JPEG mode is set in the selectors 27 and 29, the reproduced image data is stored in the frame memory 30 without passing through the color converter 28, and the image output device 31 stores this data. Print out or memorize.

As described above, according to this embodiment, the background color and the designated number of representative colors are extracted, the background color and the representative color are encoded, and the background which approximates the input image data is extracted. Encode after replacing with a color or a representative color. Also, the information about the background color and the representative color is decoded, and the decoded image data is replaced with the approximate background color or representative color and output. Therefore, by replacing the image data of a color original, which originally has a limited number of colors, with the data of the background color or the representative color, noise in the image can be removed and edges can be made clear, thus reducing redundancy. As a result, the coding efficiency can be improved, and the mosquito noise of the decoded image can be removed to prevent the deterioration of the image quality.

Further, the coded data generated in this embodiment can be decoded as it is even in a decoding device conforming to a standard coding system such as JPEG.

[0035]

Second Embodiment An image processing apparatus according to the second embodiment of the present invention will be described below. It should be noted that in the second embodiment, substantially the same configurations as those in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

FIG. 5 is a block diagram showing an example of the configuration of the encoding unit of the second embodiment according to the present invention, but a configuration different from the encoding unit shown in FIG. 3 will be described. Reference numeral 101 denotes an achromatic color extractor, which changes the output destination depending on whether the input image data is an achromatic color or a chromatic color. A representative color extractor 102 extracts a representative color from the input image data by the LBG method. The LBG method is a method generally used for generating a codebook in vector quantization. 105
Is a representative achromatic color extractor, which extracts an achromatic representative color from the input image data. A color converter 103 converts the input image data according to the extracted background color, representative color and representative achromatic color. 104 is a color encoder for the background color,
The representative color and the representative achromatic color are encoded.

The representative color extractor 102 inputs the background color from the background color extractor 3 and the representative color number from the representative color number designator 4,
Image data from the frame memory 2 is input to the achromatic color extractor 101.

The achromatic color extractor 101 first converts the background color input from the background color extractor 3 into L * a * b * color space image data (this data is referred to as Lb, ab, bb), and then, , Convert the input image data input from frame memory 2 to L * a * b * color space data (let this data be L, a, b),
When the expressions (1) and (2) are satisfied, the image data is output to the representative color extractor 102. If only the equation (1) is satisfied, the image data is output to the representative achromatic color extractor 105.
If expression (1) is not satisfied, nothing is output. √ {(Lb-L) ^ 2 + (ab-a) ^ 2 + (bb-b) ^ 2} ≧ Th4… (1) √ {a ^ 2 + b ^ 2} ≧ Th5… (2) where x ^ 2 represents the square of x

The representative color extractor 102 creates a histogram from the input chromatic color image data, selects a designated number of representative color candidates from the histogram as in the first embodiment, and then uses the LBG method. To determine the representative color. The LBG method is a method in which a space centering on a representative color candidate is divided and the centroid of the divided space is set as a new representative color candidate. Then, the space division and the movement of the center of gravity point are repeated, and if the distance of movement of the center of gravity point becomes equal to or less than a predetermined threshold value, it is assumed that the distance has converged, and the center point of gravity at that time is set as the representative color.

Similarly, the representative achromatic color extractor 105 creates a histogram of the lightness L * data from the input achromatic color image data and selects four representative achromatic color candidates in descending order of appearance frequency. And LBG is also the representative achromatic candidate
Give the law. The threshold value Th6 in which the distance of the representative achromatic color is set in advance
Those smaller than are merged into the larger of the split spaces. As a result, the number of representative achromatic colors and the representative achromatic color are determined.

After the representative color and the representative achromatic color have been extracted in this way, the background color is first extracted from the background color extractor 3 by the color encoder 1.
Input to 04. The color encoder 104 first generates a representative color start code indicating the beginning of the representative color code data and outputs the representative color start code to the output device 11, and then outputs the L * a * b * color space data of the background color to the output device 11. Output. Next, the representative achromatic color extractor 105
The number of representative colors and L * data of each representative achromatic color are input from and output to the output device 11. Next, the representative color number designator 4
Enter the number of representative colors from the representative color extractor 5
The * a * b * color space data is input, differential encoding is performed on them, and output to the output device 11. After that, a JPEG start code indicating the start of the JPEG code data is generated, and the output device 11
Output to.

Subsequently, the encoder 9 outputs the header data defined by JPEG to the output device 11, and the encoding of the image data is started. The color converter 6 first sets the background color and L of each representative color.
The * a * b * color space data and the representative achromatic L * data are input, and then the pixels forming a block of a predetermined size are sequentially read from the frame memory 2. Input pixel data to L * a *
After the conversion into the b * color space data, when the expressions (1) and (2) are satisfied, the color difference between the pixel data and each representative color is compared, and the representative color having the smallest color difference is replaced. If only equation (1) is satisfied, the a * and b * values of the pixel data are set to zero and L *
And the representative achromatic color are calculated and replaced with the nearest representative achromatic color. Then, when the replacement of all pixel data of the block is completed, the data of the block is output to the DCT unit 7.

On the other hand, the configuration of the decoding unit is the same as that of FIG. 4, but the processes of the color decoder 23 and the color converter 28 are different.

First, the case where the encoding mode is the representative color mode will be described. The color decoder 23 uses the background color L * a * b *
After decoding the color space data, the number of representative achromatic colors is decoded and the number of L * data (representative achromatic color) of that number is decoded, and then the number of representative colors is decoded and the number of L * of that number is decoded. Decode a * b * color space data (representative color). Each decoded data is input to the color converter 28 and set.

Then, the color converter 28 inputs the pixel data forming the blocks sequentially output from the IDCT device 26,
After converting to L * a * b * color space data, the pixel data
When (1) and (2) are satisfied, the color difference between the pixel data and each representative color is compared, and the representative color having the smallest color difference is replaced. When only equation (1) is satisfied, the values of a * and b * are set to zero, the distance between L * and the representative achromatic color is calculated, and the nearest representative achromatic color is replaced. If the expression (1) is not satisfied, the background color is replaced.

When the encoding mode is the JPEG mode, it is as described in the first embodiment.

As described above, according to this embodiment, the same effect as that of the first embodiment can be expected, and since the representative color is set for the achromatic color as well, the color image including the achromatic color or the color close to the achromatic color is obtained. Effective processing can be applied to images as well.

[0048]

Third Embodiment An image processing apparatus according to the third embodiment of the present invention will be described below. In addition, in the third embodiment, substantially the same configurations as those in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

FIG. 6 is a block diagram showing an example of the configuration of the encoding unit of the third embodiment according to the present invention, but a configuration different from the encoding unit shown in FIG. 3 will be described. An image processing unit 301 displays the image stored in the frame memory 2 and the cursor of the pointing device 305 on the display 302. Reference numeral 303 is a background color buffer that stores background colors, 304 is a representative color buffer that stores representative colors, and 306 is a counter that counts the number of designated representative colors. The pointing device 305 is a mouse, a trackball, a touch panel, or the like.

When the image data is stored in the frame memory 2, the image processing section 301 displays the image on the display 302. FIG. 7 shows this display state. Area of Figure 7
At 320, the image stored in the frame memory 2 and the cursor 321 of the pointing device 305 are displayed. Further, a background color panel 322 displaying a background color, a representative color panel 323 displaying a representative color, and an “end” button 324 for inputting the end of the representative color designation are displayed.

The user specifies the background color panel 322 based on the display of FIG. 7 to select the background color from the image displayed in the area 320. Needless to say, these designations and selections are performed by the pointing device 305. The image processing unit 301 reads out the image data of the selected background color from the frame memory 2, stores it in the background color buffer 303, and displays it on the background color panel 322. When the user specifies the representative color panel 323, the image processing unit 301
The counter 306 is cleared and waits for the user to select the representative color. When the user selects the representative color, the image data is read from the frame memory 2, stored in the representative color buffer 304, displayed on the background color panel 323, and the counter 306 is incremented.

When the user who has designated the necessary representative colors has selected the "End" button 324, the background color and the representative colors are fixed. After that, the first implementation is that the background color is input from the background color buffer 303 instead of the background color extractor 3 and the representative color is input from the representative color buffer 304 instead of the representative color extractor 5 to the color encoder 10. Only different from the example.

As described above, according to this embodiment, the same effect as that of the first embodiment can be expected, and the user can arbitrarily select the background color and the representative color.

[0054]

[Modifications] In each of the above-described embodiments, an example in which color replacement is performed using color differences in a uniform color space has been described, but the present invention is not limited to this, and distance conversion is performed by other conversions. You may ask.

Although JPEG coding has been taken as an example of coding, the present invention is not limited to this, and other coding can be adopted. For example, the coding may be a combination of Hadamard transform and vector quantization.

Furthermore, the unit of encoding and processing is not limited to the above, and for example, the processing can be divided in units of a predetermined number of lines.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device.

Needless to say, the present invention can also be applied to the case where it is achieved by supplying a program to a system or an apparatus.

[0059]

As described above, according to the present invention,
Taking advantage of the fact that color originals are often expressed with a limited number of colors, an image processing apparatus that improves coding efficiency by removing noise from a color image and clarifying edges A method can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an encoding unit of an image processing apparatus that performs JPEG encoding.

FIG. 2 is a block diagram showing the configuration of a decoding unit of an image processing apparatus that performs JPEG encoding.

FIG. 3 is a block diagram showing a configuration example of an encoding unit of the image processing apparatus according to the present invention,

FIG. 4 is a block diagram showing a configuration example of a decoding unit of the image processing apparatus according to the present invention,

FIG. 5 is a block diagram showing a configuration example of an encoding unit according to a second embodiment of the present invention,

[Fig. 6] Fig. 6 is a block diagram illustrating a configuration example of an encoding unit according to a third embodiment of the present invention.

FIG. 7 is a diagram showing an example of a display state of a display.

[Explanation of symbols]

 3 Background color extractor 4 Representative color number designator 5 Representative color extractor 6 Color converter 10 Color encoder 22 Start code detector 23 Color decoder 28 Color converter 101 Achromatic color extractor 102 Representative color extractor 103 colors Converter 104 Color encoder 105 Representative achromatic color extractor 301 Image processing unit 303 Background color buffer 304 Representative color buffer 306 Counter

Claims (8)

[Claims] 1. An image processing apparatus for encoding input image data, comprising: designating means for designating the number of colors; and extracting from the image data representative colors of the number designated by the designating means. Extraction means, first encoding means for encoding the information of the representative color extracted by the extraction means, conversion means for converting the image data based on the representative color, and conversion by the conversion means An image processing apparatus comprising: a second encoding unit that encodes image data. 2. An image processing apparatus for encoding input image data, comprising: extracting means for extracting an arbitrary number of representative colors from the image data; and a representative color of the representative colors extracted by the extracting means. A first encoding means for encoding information, a converting means for converting the image data based on the representative color, and a second encoding means for encoding the image data converted by the converting means. An image processing apparatus having. 3. The image processing apparatus according to claim 1, wherein the conversion unit reduces the number of colors of the image data to the number of the representative colors. 4. The converting means converts the image data into a uniform color space and replaces each pixel of the image data with a closest representative color in the color space.
The image processing apparatus according to claim 1 or claim 2. 5. An image processing apparatus for decoding coded image data, comprising: first decoding means for decoding information of a representative color included in the code data; and image data included in the code data. An image characterized by having a second decoding means for decoding and a conversion means for converting the image data decoded by the second decoding means based on the representative color decoded by the first decoding means Processing equipment. 6. An image processing method for encoding image data, comprising a designating step of designating the number of colors, and an extracting step of extracting from the image data the number of representative colors designated in the designating step. A first encoding step of encoding the information of the representative color extracted in the extracting step, a converting step of converting the image data based on the representative color, and encoding the image data converted in the converting step. An image processing method comprising: a second encoding step. 7. An image processing method for encoding image data, comprising: an extracting step of extracting an arbitrary number of representative colors from the image data; and encoding information of the representative colors extracted in the extracting step. A first encoding step, a conversion step of converting the image data based on the representative color, and a second encoding step of encoding the image data converted in the conversion step, Image processing method. 8. An image processing method for decoding encoded image data, comprising: a first decoding step of decoding information of a representative color included in the encoded data; and an image data included in the encoded data. An image characterized by comprising a second decoding step of decoding, and a conversion step of converting the image data decoded in the second decoding step based on the representative color decoded in the first decoding step. Processing method.