JP2608284B2 - Image processing device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus that encodes brightness information representing a color image and color information.

2. Description of the Related Art Conventionally, there is a color image encoding apparatus that decomposes color image data into brightness information and color information and encodes each.

[Problems to be Solved by the Invention] However, in the conventional encoding method, when an image with a high resolution is encoded, encoding that reduces the amount of color information of the color image data is performed. Edges are not efficiently reproduced.

Conversely, when a halftone image is encoded, the encoding efficiency is reduced unless encoding is performed such that the amount of color information of the color image data is reduced.

Further, when decoding color image data, if the data amount of a data unit including coded data differs for each predetermined image unit, there is a problem that it is difficult to handle the data.

The present invention has been proposed in order to eliminate the above-mentioned disadvantages of the conventional example. The purpose of the present invention is to efficiently encode image brightness information and color information while suppressing visual deterioration. It is an object of the present invention to provide an image processing apparatus which can perform color image data decoding into a data unit which is easy to handle, for example, when decoding color image data.

In particular, an object of the present invention is to propose an image processing apparatus that performs encoding with good color reproducibility regardless of the type of an image to be encoded.

According to an embodiment of the present invention, there is provided an image processing apparatus for processing an image divided into a plurality of blocks, comprising: First generating means for generating first information representing an average value of brightness in the block using the pixel in the block; and a distribution state of pixel values for brightness in the block using pixels in the block. Using a pixel in the block and the second information, excluding the first information, and A third generation unit that generates third information that can represent the brightness of each pixel; and a fourth generation unit that generates fourth information indicating a color of the block using pixels in the block. 4 generating means, and for each block, Forming means for forming independent first to fourth codes respectively corresponding to the first information to the fourth information, the forming means comprising: In the case of the first state that includes the edge of
If the third information for the block is encoded with a long code and the fourth information is encoded with a short code, and the second information indicates a second state different from the first state, , And operates to encode the third information for the block with a short code and to encode the fourth information with a long code, and a data unit including the first to fourth codes. Is set to a predetermined amount.

Embodiment An embodiment according to the present invention will be described below with reference to the accompanying drawings of FIGS. 1 to 9B.

<Pixel Block> In this embodiment, for simplicity of description, the size of the pixel block is 2 × 2 pixels. FIG. 2 is a diagram showing the division of the pixel block. In FIG. 2, the image is 4
It is divided into blocks (2a to 2d). 1 indicates a color image. The color pixel 1 is the color stimulus R
(Red), G (Green), and B (Blue) information are provided as a plurality of bits of information (hereinafter, referred to as multi-valued information). Here, for convenience of explanation, each of the R, G, and B signals is assumed to be 8 bits. Therefore, one color pixel has a total information amount of 24 bits. FIG. 3 shows the configuration of such a pixel in a block. 3, 3a to 3d represent one color pixel in FIG. Hereinafter, the color pixel 3 in this pixel block
a, 3b, 3c, 3d the color image data respectively in the order of, X _11, X _12, X
₂₁ and X ₂₂ . X ₁₁ to X ₂₂ are as previously described R of each pixel, G, B
Is multi-valued information. Let the information of R of X _{11 be} R _11, and similarly, G,
If B is G ₁₁ and B ₁₁ and X ₁₂ , X ₂₁ and X ₂₂ are the same, X ₁₁ = 11R ₁₁ , G ₁₁ , B _{11 11} and X ₁₂ = ｛R ₁₂ , G ₁₂ , B ₁₂ ｝, X ₂₁ = ｛R ₂₁ , G ₂₁ , B ₂₁ ｝ and X ₂₂ = R ₂₂ , G ₂₂ , B ₂₂ ｝.

FIG. 1 is an overall block diagram of an encoding device according to an embodiment. This encoding device encodes each block with four pixels as one block, and performs encoding based on the characteristics of the block.
This is converted into an encoded code as shown in FIG. 9A or FIG. 9B. In the encoding process, the RGB signal is converted by the ROM 4 into a Y (brightness) IQ (color difference) signal, and the converted YIQ signal is extracted by the arithmetic units 8, 9, and 10 for each signal. And, depending on its characteristics, two encodings by encoders 11 and 12 (encoder 11 encodes according to the format of FIG. 9A, and encoder 12 encodes according to the format of FIG. 9B). One of the encoded codes is output from the synthesizer 13.

FIG. 4 shows the outline of the encoding according to the embodiment together with FIGS. 9A and 9B. In the figure, a decoding method is also shown to make it easier to understand the characteristics of the coding. m _Y indicates the average brightness in the block. σ _Y is edge information on the brightness in the block (in the present embodiment, the standard deviation of Y _ij is used as edge information). N _ij is a value obtained by standardizing a change in brightness from the block center (having brightness m _Y ) in each pixel direction by the standard deviation σ _Y , that is, N _ij = (Y _ij −m _Y ) / σ _Y. Accordance connexion, by _{decoding, Y ij '= N ij ·} σ Y + m Y I i' = m I I Q '= m Q is obtained. The circuit of the embodiment shown in FIG. 1 is configured to realize the above encoding.

<RGB → YIQ Conversion> Returning to FIG. 1, reference numeral 4 denotes a signal conversion read-only memory (hereinafter abbreviated as ROM) for converting RGB information into YIQ information with reference to a table. I and Q are luminance information Y and color difference information I and Q generally used as TV signals. This signal conversion ROM 4 uses R, G, B information as an input address and outputs Y, I, Q information as output data.
Therefore, the Y-information calculated from X ₁₁ Y _11, I information I _11, Q
Information is referred to as Q _11. Hereinafter, the same applies to X ₁₂ , X ₂₁ , and X ₂₂ .

Reference numerals 5, 6, and 7 denote registers for accumulating the Y, I, and Q information of the I block converted by the signal conversion ROM 4 for each YIQ. That is, 5 is a Y information register, 6 is an I information register, 7 is Q
It is an information register. These registers 5, 6, and 7 have X ₁₁ , X
_12, X _21, Y calculated from _{X 22 11 ~Y 22, I 11} ~I 22, Q 11 ~Q 22
Information is taken in order.

All information X ₁₁ to X ₂₂ as information for <feature extraction> block, after being stored in the register 5-7 is input to the arithmetic unit 8-10 connected to the register. The configurations of these arithmetic units 8 to 10 are as shown in FIG. Here, the arithmetic unit 8 has the circuit configuration shown in FIG. 5, and the arithmetic units 9 and 10 are equivalent to the configuration of the portion 34 indicated by a broken line including the average value calculator 22 in FIG.

The arithmetic unit 8 calculates the average value of the signal Y in the block.
m _Y , σ _Y indicating edge information about brightness, and a value obtained by standardizing the change in brightness of Y ₁₁ to Y ₂₂ of each pixel with the σ _Y from the average value m _{Y in} each pixel direction.
Arithmetic units 9 and 10 each calculate the average value in the block of I information.
An average value m _Q in the block of m _I and Q information is calculated.

The arithmetic unit will be described with reference to FIG. 5 by taking as an example an arithmetic unit 8 for extracting a feature of Y information. Terminal 18 inputs the Y ₁₁ information, the terminal 19 is Y _12, the terminal 20 is Y _21, the terminal 21 inputs the Y ₂₂ information. The average value calculator 22 adds all of these pieces of information, and divides the sum by the number of input information (four). In this case, the lower two bits of the output of the average value calculator 22 are shifted off to the lower side to perform division by 4.
Therefore, the output m _Y of the average calculator 22 is Becomes

As described above, the arithmetic units 9 and 10 use a circuit composed only of the portion surrounded by the broken line 34 in FIG. 5, and output the average value m _I in the block for I and Q from the output terminal 28 thereof. , m _Q is output.

Next, the edge information amount σ _Y regarding the brightness calculated by the arithmetic unit 8 will be described. In this example,
The standard deviation value in the block is used as the edge information amount σ _Y regarding the brightness. To determine the standard deviation sigma _Y, subtractor 23, ROM24,26, and the average value calculator 25 are used. Σ _Y output from ROM 26 is (Where i = 1,2, j = 1,2).

That is, subtractor brightness of the pixel in the 23 information Y ₁₁ to Y ₂₂ is input as minuend value, the average value m _Y in the block is input as a subtraction value. Accordingly, the subtractor 23 outputs a difference value S _ij from the average value m _Y of the brightness information Y _ij of each pixel. That is, S _ij = Y _ij −m _Y {i = 1,2, j = 1,2}. These difference values S _ij is input to the ROM 24. ROM24
Is a ROM for outputting the square of the input. The output of the ROM 24 is the square of the difference value S _ij of each pixel, that is, S _ij ² = (Y _ij −m _Y ) ² {i = 1,2, j = 1,2}. The square value of the difference value S _ij of each pixel is input to the average value calculator 25 and the output σ _Y ² is Which is the variance within the block. This result is RO
Input to M26. ROM 26 outputs the square root of the input. That is, Is output. This represents the standard deviation value within the block. This is output from the terminal 29 as the edge information amount σ _Y regarding the brightness. As the ROMs 24 and 26, a LUT (look-up table) ROM or the like may be used.

In the embodiment, the value of σ _Y is used as a guide as to whether or not an edge exists in a block.

It will now be described a method for normalizing the Y information Y ₁₁ to Y _22. The output of the aforementioned subtracter 23 is the difference value S ₁₁ to S ₂₂ of the mean value m _Y and each Y ₁₁ to Y _22. At divider 27 divides these difference values in the standard deviation sigma _Y. That is, if this quotient is N _ij , It is. However, when the standard deviation σ _Y is “0”, N _ij
Are all 0. σ _Y is a standard deviation, that is, an average variation of each pixel. N _ij divided by S _ij with this σ _Y
Is considered to indicate the gradient of brightness in the direction of each pixel from the center position of the block, as shown in FIG. S _ij
Is considered to be a gradient, but by normalizing with σ _Y ,
N _ij is a quantity indicating a gradient independent of the size of the block. Incidentally, N _ij In view of the fact YIQ is respectively 8-bit, 9
Bit length.

The standard value C ₁₁ in the Y ₁₁ from the terminal 30, or less, from the terminal 31
Standard value C ₁₂ in the Y _12, the standard value C ₂₁ in the Y ₂₁ from the terminal 32, the terminal
From 33 outputs the standard value C ₂₂ in the Y _22.

<Encoding> The outputs m _Y , σ _Y , and N _ij of the arithmetic unit 8 are input to the encoder 11, and the outputs m _I , m _{Q of the} arithmetic units 9 and 10 and the σ _Y are encoded. Is input to the device 12 (see FIG. 1).

FIG. 6 shows a block diagram of the encoder 11 for brightness information.
The encoder 11 comprises a ROM 41, a ROM 42 and a selector 43. Terminals 37 to 40 are provided with σ _Y ,
N _ij is input, and a similar edge information amount σ
_Y inputs. ROM41 and ROM42 have input standardized values N _{11 respectively.}
The to N ₂₂ quantizes the I connexion speed stage of its size. As described above, when the R, G, B are each 8 bits, Y, I, Q information also has information about each 8 bits and the normalized value N ₁₁ to N ₂₂ nine-bit (= 512 steps) Each has the amount of information. Therefore, each normalized value N _ij is equally divided and quantized into 4 bits (= 16 steps) in the ROM 41, and is equally divided and quantized into 3 bits (= 8 steps) in the ROM 42 to reduce the information amount. The reason why the degree of quantization is changed is that there is no problem even if the quantization is coarse when the block does not include a structure such as an edge. Therefore, the output from the ROM 41 is N ₁₁ ,
N ₁₂ , N ₂₁ , and N _{22 have} a 16-bit length in the order, and the output from the ROM 42 has a 12-bit length.

The output of the ROM 41 is sent to the selector 43. On the other hand, the 12 bits of the output of the ROM 42 further have 4-bit “00” as lower bits.
00 "is transmitted to the selector 43.
Selectively outputs the outputs of the ROM 41 and the ROM 42 in accordance with the magnitude of the edge information amount σ _Y regarding the brightness. That is,
Assuming that T ₁ is a predetermined threshold value, N _ij : 4-bit quantization (σ _Y > T ₁ ) N _ij : 3-bit quantization (σ _Y <T ₁ ). Performing different quantization depending on the magnitude of σ _Y in this way is because when it is determined based on the brightness information Y that there is a large change in brightness in the block, information related to the brightness ( In particular, when the quantization density of N _ij ) is finely determined, and conversely, when it is determined that there is no large change in brightness in the block, the information amount of the brightness information is reduced, so that the color information is consequently reduced. This is because it is possible to increase the amount of information.

The edge information amount σ _Y (7 bits) regarding the brightness is added to the most significant bit portion of the output of the selector 43 selected in this manner, and further below that, the average of the brightness information in the block is averaged. The value m _Y (8 bits) is added and output from the terminal 44 as brightness information (31 bits in total).

In FIG. 6, a symbol such as “MSB∨LSB” represents a shifter. For example, the shifter 63 converts the MSB side input data into MSB.
The input data on the LSB input side is arranged in order from the MSB side input data.

Next, FIG. 7 shows a block diagram of the encoder 12 (FIG. 1) for encoding the color information (I, Q).

Terminal 46 inputs the average value m _I in the block of I information outputted from the arithmetic unit 9 (Figure 1). Similarly, a terminal 47 inputs an average value m _Q in the block of Q information. One input of the selector 48 is obtained by combining m _I and m _Q in the order of I and Q.
Since m _I and m _Q are each 8 bits in size, the combined one is 16 bits. On the other hand, the other inputs of the selector 48 are shifted by the shifters 64 and 65 to the average values m _I and m _Q , respectively, in the lower two positions.
The bits deleted one by one are combined by the shifter 66 in the order of I and Q from the upper bits, and the shifter 67 adds 4-bit “0” to the upper bits. The selector 48 Yotsute selectively outputs these two inputs to the size of the edge information amount sigma _Y. That is, the average value
m _I and m _Q are quantized into 6 bits (if σ _Y > T ₁ ) and 8 bits (if σ _Y <T ₁ ). If the magnitude of the edge information amount σ _Y is larger than the above-described predetermined threshold T ₁ (if σ _Y > T ₁ ), the latter input, that is, the higher order of the 6-bit average values m _I and m _Q 4 more
Select the one with the bit "0" added. Conversely, if the magnitude of σ _Y is smaller than the threshold T ₁ , the former input, that is, a combination of the 8-bit average values m _I and m _Q as they are, is selected. In other words, when it is determined based on the brightness information that a large change in brightness exists in the block, the information amount of the color information is reduced, and when it is determined that the block does not exist, the information amount of the color information is reduced. To increase.

The output of the selector 48 selected in this way is
This is one input of the synthesizer 13 shown in FIG.

<Synthesis> The outputs of the encoders 11 and 12 are input to the synthesizer 13. FIG. 8 shows the configuration of the synthesizer 13. In the figure, a terminal 50 inputs the encoded brightness information from the encoder 11. The terminal 51 receives the encoded color information from the encoder 12. The shifter 70 extracts the lower 4 bits from the brightness information, and the shifter 71 extracts the upper 4 bits from the color information. The OR circuit 52 calculates the logical sum of the 4 bits from the upper bits.

That is, when the magnitude of the edge information amount σ _Y regarding the brightness is larger than the threshold value T ₁ , significant information of the brightness information is contained in 4 bits of the LSB output of the shifter 70, and Since "0" is entered in the four bits of the MSB output, the output of the OR circuit 52 contains significant information (4
Bit length). Conversely, when σ _Y is smaller than the threshold value T ₁ , “0” is output to the LSB output of the shifter 70 and the shifter 71
Because significant information of color information is included in the MSB output of
The output of R circuit 52 is 4-bit significant information about the color.

In the shifter 72, the information (27 bits) excluding the lower 4 bits of the brightness information is set as the highest order, then the output (4 bits) of the OR circuit 52 is arranged, and the upper 4 bits of the color information are set at the lowest order. The removed information (12 bits) is arranged and output from the terminal 53, thereby completing the encoding.

FIGS. 9A and 9B show the arrangement of the 43 bits thus encoded. Figure 9A if the edge information amount sigma _Y is greater than thresholds T ₁ Katsuta, FIG. 9B is sigma _Y said thresholds T ₁
This is the arrangement of the bits of the code in the case of being smaller. or,
The numbers on the top of the figure indicate the bit positions.

<Decoding> Further, the decoding of the code according to the embodiment may be performed by reversing the above-described procedure. That is, 37 to 42 most significant bits of the input code to (sigma _Y) is taken out, as compared with the thresholds T ₁ of said, check whether those containing large changes in block brightness,
It is determined whether the code bit arrangement is as shown in FIG. 9A or FIG. 9B. Next, take out the 36-28 th bit as the mean value m _Y of Y information.

Further, N ₁₁ to N ₂₂ the Y information Y ₁₁ to Y ₂₂ of each pixel normalized
Is obtained by extracting four bits at a time when it is determined that there is a large change in brightness in the block (FIG. 9A), and is obtained by extracting three bits at a time when it is determined that there is no change (FIG. 9B). For the remaining bits, these are divided into two,
The average value m _I in the block of information and the lower order are obtained as the average value m _Q in the block of Q information.

Therefore, if Y information, I information and Q information to be reproduced are Y ₁₁ ′ to Y ₂₂ ′, I ₁₁ ′ to I ₂₂ ′ and Q ₁₁ ′ to Q ₂₂ ′ for each pixel, then Y _ij ′ = N _{ij × σ Y + m Y I} ij = m I Q ij = m Q where, i = 1, 2, a j = 1, 2.

Furthermore, as in the case of encoding, R information reproduced using a ROM for converting YIQ information into RGB information with reference to a table is used.
GB information R ', G', B 'is obtained. In this manner, decoding may be performed by the reverse procedure of encoding.

<Modification of Embodiment> In the above embodiment, the standard deviation value in the block was used as the edge information amount for brightness, but the difference between the maximum value and the minimum value of the pixel values in the block was also used. Also, the circuit configuration can be simplified. In the embodiment, the Y, I, Q information of the TV system is used as the information format for encoding. However, the information format for decomposing into other brightness information and color information, for example, CIE1976 (L ^* , a ^* , b ^* ) L ^* , a ^* , b ^* information of the space may of course be used. Further, in the embodiment, thinning allocation of bits is used as an encoder for simplicity of description, but other encoding, for example, vector quantization encoding may be used.

<Effects of Embodiment> According to this embodiment, the information amount per block was 96 bits before coding, but this is coded to 43 bits to achieve a compression ratio of 44.8%. Furthermore, when brightness information and color information are independently subjected to fixed-length encoding without performing encoding according to the presence or absence of an edge, brightness information 31 bits, color information 16 bits, and a total of 47 bits are required. By performing encoding in accordance with the presence or absence of an edge, this can be further reduced to 4 bits, and encoding with less visual deterioration can be performed.

Thus, when a color image is divided into a plurality of pixel blocks, and the brightness information and the color information are decomposed in units of blocks, and a fixed-length encoding is performed as a whole, the brightness that is easily generated during the encoding is as follows: By changing the magnitude of the information amounts of the brightness information and the color information based on the information of the edge information amount (for example, standard deviation) of Utilizing the visual characteristics of masking, there is no visual deterioration,
Encoding that provides high efficiency can be performed.

: In addition, since the encoding is of fixed length in block units, when the image data is stored in an image memory or the like, it is possible to maintain the positional relationship of the pixels in the image. Therefore, the required amount of memory can be kept constant depending on the size of the image, and even when performing image processing, it is possible to easily extract information on pixels near the periphery.

In other words, in the embodiment, a fixed-length codeword indicating that an edge is detected in a block is generated, and the brightness information and the color information are respectively determined by predetermined bits having different lengths according to the presence or absence of the edge. A configuration for encoding to a long code is disclosed.

With this configuration, the brightness information (Y or L ^* ) and the color information (IQ or a ^* b ^* ) are converted according to the degree of the edge in FIG. 9A, FIG. 16A, or FIG. 9B or FIG. 16B. In this way, the bit length (quantization density) is encoded to be short (coarse). For this reason, the compression efficiency is increased, and at the same time, a compressed image matching the human visual characteristics can be obtained.

Fixed length code word which indicates that the edge is detected, for example, in the embodiment is a fixed length of sigma _Y.

If the total length (43 bits in the embodiment) of the encoded code corresponding to one block of color image data is fixed regardless of the presence or absence of an edge, when the image is stored in the above-described image memory or the like, There are effects such as the positional relationship between pixels in an image can be maintained.

In the edge detection, a brightness edge is detected from brightness information.

The brightness portion of the coded code is standard deviation (σ _Y ), average brightness (m _Y ), gradient in the block of brightness (N _ij or C
_ij ) is encoded (quantized) to a predetermined length, and the color portion is a color average value (m _I , m _Q , m _a , m _b ) or a color gradient (E _I , E _Q , m _a1 , m
_a2 , _mb1 , _mb2 ) are encoded (quantized) to a predetermined length.

In the embodiment, a fixed-length codeword indicating that an edge is detected in a block is generated, and information indicating the distribution configuration of the color image data in the block and the average value of the color image data in each distribution region are calculated as follows. A configuration is disclosed in which a code word having a predetermined bit length of a different length is encoded according to the presence or absence of an edge.

With this configuration, information on edges is efficiently compressed and stored, and the decoded image also matches human visual characteristics.

[Effects of the Invention] As described above, according to the image processing apparatus of the present invention,
When encoding the brightness information and the color information of the color image, the second information indicating the distribution state of the pixel values in the block indicates whether or not this block includes an edge image. The first information, which is the average value of the brightness of the pixels, and the second information indicating the distribution state of the pixel values in the block
Since the code lengths of the third information that enables the brightness of each pixel in the block to be represented using the information of the above and the fourth information that indicates the color of the block are different, the efficiency according to the type of the image is different. Encoding can be performed.

Further, since the second information indicating the distribution state of the pixel values in the block is a code independent of the third information, the code information output from the image processing apparatus is used to correspond to the feature of the image. Even when processing is performed, the features of the image can be easily extracted.

Also, the sign of the first information, the sign of the information of the second information,
Since the data amount of the data unit including the code of the third information and the code of the fourth information is set to a predetermined value, handling becomes easy, for example, when decoding color image data.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of an encoding circuit according to an embodiment, FIGS. 2 and 3 are diagrams illustrating pixel blocks used in the embodiment, and FIG. 4 is an encoding result and a decoding result in the embodiment. FIG. 5 is a block diagram showing a configuration of an arithmetic unit of the embodiment, FIG. 6 is a block diagram showing a configuration of an encoder 11 of the embodiment, and FIG. 7 is a configuration of an encoder 12 of the embodiment. FIG. 8 is a block diagram showing a configuration of a combiner 13 in the embodiment, and FIGS. 9A and 9B are block diagrams showing a configuration of a format of an encoded code in the embodiment. In the figure, 8, 9, 10,... Arithmetic units, 11, 12,.

Claims (1)

(57) [Claims] 1. An image processing apparatus for processing an image divided into a plurality of blocks, comprising: generating first information representing an average value of brightness in the block using pixels in the block; Generating means for generating, using pixels in the block, second information indicating a distribution state of pixel values of brightness in the block; and pixels in the block; Using the second information,
Third information that is information excluding the first information and that can represent the brightness of each pixel in the block,
A third generating unit that generates, a fourth generating unit that generates fourth information indicating a color of the block using pixels in the block, and the first information to the fourth information for each block. Forming means for forming independent first to fourth codes respectively corresponding to four information, wherein the forming means is such that the block includes a predetermined amount of edges in the second information. In the case of the first state shown, the third information for the block is encoded with a long code, the fourth information is encoded with a short code, and the second information is encoded with the second code.
If the information of the second information indicates a second state different from the first state, the third information for the block is encoded with a short code and the fourth information is encoded with a long code. An image processing apparatus that operates and sets a data amount of a data unit including the first code to the fourth code to a predetermined amount.