JP2585271B2 - Color image processing method - Google Patents

Description

The present invention relates to a color image processing method used in a color image input device that reads a color image as an electric signal.

(Prior Art) Conventionally, as a color image sensor used for reading a color image, a so-called dot sequential sensor in which ordinary three color filters are sequentially arranged on one line sensor (Japanese Patent Application No. 58-178659). It has been known. The dot-sequential sensor has an advantage that color information can be read by one line sensor, but cannot obtain three color signals at the same position on the document. Since many color signal processing systems and output systems require three color signals for one pixel,
The signals need to be matched. Normally, as shown in FIG. 2, every three adjacent elements are regarded as the same pixel as a group, and the output signals of these three elements are set as three color signals of one pixel. This method requires three times as many elements as the number of read pixels. Although there is no problem when reading an image having a smooth change in density, such as a photograph, color noise, that is, a hue shift occurs in a portion of the document where the brightness changes suddenly, resulting in a high-definition image. There is a problem in reading the image.

In order to remove color noise, there is a method of equivalently shifting the read center of each color signal by interpolation (see Japanese Patent Application No.
274128). Although this method has a great effect on removing color noise, on the other hand, since the interpolation operation is a kind of averaging process, by replacing the signal obtained by the sensor with the interpolated signal, the resolution of the signal is reduced. .

There is also a method in which each light receiving element is regarded as one pixel, and a color signal which does not correspond to each light receiving element is obtained by performing an interpolation operation on color signals on both sides of the light receiving element. "Quality", the 12th Image Electronics University (Showa 58)). However, with a simple interpolation operation, the resolution does not substantially increase, and only a signal having a lower resolution than the signal density can be obtained.

(Problems to be Solved by the Invention) As described above, when a method of regarding a set of three light receiving elements as one pixel is used, three lines are required for the line sensor as compared with the case of monochrome.
Twice as many light receiving elements are required. For example, to read A4 width at a density of 16 lines / mm, 10080 light receiving elements are required, and a large-scale licensor must be used. At the same time, there arise problems such as an increase in the speed of the sensor output signal and an increase in the amount of light required for illumination. There is also a problem that color noise occurs as described above. Color noise can be removed by simple interpolation, but on the other hand, it reduces the resolution of the signal. Further, in the method in which each light receiving element is regarded as one pixel, the resolution obtained is lower than the signal density. As described above, in the conventional method, the number of light receiving elements equal to or more than three times the resolution is required for the color sensor, and the characteristics of the dot sequential color sensor are not fully utilized.

[Configuration of the Invention] (Means for Solving the Problems) In the present invention, from the color signals read by the color sensor, each light receiving element unit of the line sensor is considered as one pixel, and the color signals of three colors for each pixel are considered. , And another color signal of the color is used in the calculation.

(Operation) In general, the three color components are not completely independent but have a certain degree of correlation with each other. Therefore, the output signal of a certain light receiving element has information on the other color components together with the color component at the position of the light receiving element. Therefore, by using a color signal other than the color signal, a signal having a higher resolution than a signal obtained by interpolation between the color signals can be obtained.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings.
FIG. 1 shows the configuration of a color image input device using the present invention. An original surface 202 is illuminated by a light source 201, and a linear region 203 on the original surface is imaged on a color line sensor 205 at an equal magnification by a close contact optical system such as a distributed index cylindrical lens array 204. As shown in FIG. 3, the color line sensor 205 sequentially arranges three color filters of red (R), green (G), and blue (B) on a light receiving element of a one-dimensional line sensor using a CCD or the like. These are arranged and called a dot-sequential color sensor. Here, the light-receiving element density of the line sensor and 2 per mm
And 4. The components of the color filters at the positions corresponding to the respective light receiving elements are converted into electric signals by this sensor, and the color image information on this line is read as an electric signal by sequentially reading the output of the line sensor (main scanning). be able to. Thereafter, the output signal of the first element first light receiving element and X _i, in particular R filter, G filter, the output signal of the B filter and R _i, G _i, B _i.

The light source, the optical system, and the line sensor are integrated with the carriage 206 and scanning (sub-scanning) is performed in a direction perpendicular to the main scanning direction, so that information on the entire original image can be read.

The output signal 251 of the line sensor is amplified by the amplifier 207, and A / A
After conversion into a digital signal 252 by the D converter 208, the normalization circuit 209 normalizes the white level and the black level. Due to variations in the sensitivity of each light receiving element of the line sensor and uneven illuminance of the light source, a uniform output signal cannot be obtained even when a document having a uniform reflectance is read. In this circuit, a white reference plate and a black reference plate at the end of the document surface are read in advance, and the obtained reference signal is stored in a line memory or the like. The variation is corrected.
Thus, for an original having the same reflectance as the white reference plate and the black reference plate, the image signals become 1 and 0, respectively, and the white balance is performed simultaneously with the correction. The specific configuration of the present circuit is described in detail in, for example, Japanese Patent Application No. 59-199263. Since the standardized image signal 253 has only one color signal of RGB for each pixel, that is, each light receiving element, it is converted into three color signals for each pixel by the next extended interpolation circuit 210. Since this circuit is the point of the present invention, the processing and configuration performed by this circuit will be described in detail below.

In this circuit, the operation shown in Expression (2) is performed, and the
RGB signals R ′ _i , G ′ _i , and B ′ _i are obtained, and are used as interpolation output signals.

However, A is an interpolation matrix and takes the following values, for example.

X _i is a vector representation of sensor output signals before and after pixel i.

X _i = (X _{i + 3} X _{i + 2} X _{i + 1} X _i X _i−1 X _i−2 X _i−3 ) ^t (4) When this expression is divided into components, for example, i = 3n In this case, equation (5) is obtained.

Here, the first term is usually the DC component 1, 2, 3 of the color component.
The coefficients are selected so that the term has a DC component of zero. That For example G _'i, the following conditions are satisfied normal.

Accordingly, the first term is considered to be a linear interpolation value of the color component, and the second and third terms are considered to be correction components due to differences between other color components.

FIG. 4 shows a specific configuration of the extended interpolation circuit. As shown in FIG. 5, the image signal input to this circuit flows on a signal line of one channel in chronological order in the arrangement of the light receiving elements of the sensor. For example, seven elements of this signal are stored in the shift register 401, and the seven output signals of each stage are multiplied by a constant Ajk by the multiplication circuit 402, respectively, and the sum of the results is obtained by the addition circuit 403. This calculation is performed three times by switching the constant Ajk per pixel (k = 0, 1, 2), and the results of each calculation are written in the order of the registers 404a, b, and c. Here, the switching of the constant Ajk is performed in the order of K = 0, 1, 2 in the case of i = 3n, in the case of i = 3n + 1, in the order of K = 2, 0, 1, and in the case of i = 3n + 2 in the order of K = 1, 2, 0.
In order. Note that the multiplication circuit may be configured by a multiplier, but since the multiplication is performed by a constant, the configuration can be more easily configured by using a lookup table. The output signal of the register 404 becomes three color signals R ′ _i , G ′ _i , and B ′ _{i of the} pixel. Next, the fact that a high-resolution color signal can be obtained by the present circuit will be described.

In a normal monochrome image, there is a correlation between the brightness of adjacent pixels. Therefore, if the brightness of a certain two pixels is known, the brightness of the pixel between them can be estimated by linear interpolation of the brightness of the two pixels. Similarly, in a color image, the R, G, and B components have a spatial correlation, but there is also a correlation between the spatial changes of the R, G, and B components. That is, among the R, G, and B components, there is a high probability that the other component is increasing in a portion where a certain component is increasing. In other words, the hue component has a stronger correlation than the luminance (brightness) component. Become. For example, as shown in FIG. 1, a portion where the R component increases (between pixels 0 and 3)
Then, there is a high probability that the other G and B components also increase, and there is a high probability that the other components also decrease in the portion where the R component decreases (between the pixels 3-6).

By the way, in the dot sequential sensor, the R signals are pixels 0, 3, 6
.., The G signal reads information of pixels 1, 4, 7,..., And the B signal reads information of pixels 2, 5, 8,. B of pixels 3 and 4
Information such as components is not directly obtained as a signal, but needs to be estimated from a read signal of a sensor. Assuming that the R, G, and B components are independent of each other, it is most probable that the estimated values of the R signals of the pixels 3 and 4 are values obtained by linear interpolation of the signals of the pixels 2 and 5. According to this method, each color component is originally 8 lines / mm.
Therefore, even if the signal density is increased by interpolation, the resolution is equivalent to 8 lines / mm. However, since there is a correlation between the R, G, and B components as described above, the estimation likelihood is higher when the changes of the R and G components are considered in estimating the pixels 3 and 4 of the B signal. Become. The R signal is convex upward at pixel 3 and G
Since the signal is upwardly convex at the pixel 4, it can be estimated that the B component is also upwardly convex at the pixels 3 and 4. More specifically, a correction obtained from the difference component between the R and G signals may be added to the value obtained by linearly interpolating the B signals of the pixels 2 and 5 as the estimated value of the B signals of the pixels 3 and 4. Thus, a signal having a resolution of 8 lines / mm or more can be obtained by performing interpolation using signals of other color components having information of 24 lines / mm.
This estimation operation is represented by the matrix operation described above, and this operation can be realized by the configuration of the extended interpolation circuit.

The qualitative description of the interpolation method according to the present invention has been described above. Next, a quantitative interpretation of the present method and how to determine the coefficients of the interpolation matrix A will be described. In the present invention, the R, G, and B components of an image are considered as a linear Gaffus signal model, and for unknown R, G, and B components, a value at which the probability density of those signals is maximum is calculated as an estimated value. This is stored and output.

The differences between adjacent pixels of each of the R, G, and B components are represented by ΔR _i , ΔG _i ,
Let ΔB _i .

These signals have an average value of 0 and have a correlation between the RGB components. Probability density P _i that these signals take values of ΔR _i , ΔG _i , ΔB _i if they follow a guffs distribution
Is Here, M is a covariance matrix of ΔR _i , ΔG _i , and ΔB _i . Here, further ignoring the correlation between adjacent pixels of ΔR _i , ΔG _i , and ΔB _i , (Assumption 1) the probability density Pall through the entire signal (i = 0 to ∞) becomes Becomes D _i that maximizes the Pall is considered highly estimate the most likelihood. Here, since the exp function is monotonically increasing, the above condition is Is the same as minimizing. Here, the covariance matrix M is further symmetric about ΔR, ΔG, ΔB, ie, ΔR,
Assume that the variances of ΔG and ΔB are equal, and that the covariances between ΔR, ΔG and ΔB are equal. (Assumption 2) It is expressed as

For the sake of simplicity of the expression, X _i , Y _i , and Z _i are defined as follows.

X _i is the output signal of the i-th pixel of the line sensor, and Y _i ,
Z _i is an unknown signal that cannot be obtained by the line sensor. Therefore the above _{P 'Y i, Y i Z} i of those partially differentiated by Z _i is the maximum probability density Solving equations that 0 is obtained. Solving this, Becomes B is an NxN matrix (N to ∞), which is a matrix having a constant value on a straight line parallel to the diagonal line and substantially zero at a portion sufficiently distant from the diagonal line.

B _ij = b _{(i, j)} if | i−j | ≫0 b _{(i, j)} ０0 (13) Therefore, ignoring the term 1i−1j ≧ 4, Similarly, Also, obviously, Therefore, when X _i , Y _i , and Z _i are collectively expressed, the expression becomes 17.

Therefore, if the interpolation matrix A is defined as Expression 18, Expression (17) results in Expression (2).

That is, by performing the processing shown in the equation (2), the optimum estimated value that maximizes the probability is output as the interpolation signal.

Here, the assumption 1 used to derive this matrix,
2 is to simplify the calculation of matrix coefficients,
It is not essential to the scheme of the present invention. Even if these assumptions are not used, the calculation formula to be saved is very complicated, but the calculation formula for estimation is the same matrix calculation. However, when the condition is not satisfied, the symmetry is no longer provided for R, G, and B, so that the interpolation matrix A has different values for R, G, and B.

In the case of such a matrix, it cannot be realized by the configuration of the present embodiment, and it is necessary to slightly complicate the arithmetic expression and the processing circuit as in the second embodiment described later.

However, usually, interpolation with sufficiently high accuracy can be performed even by using a matrix obtained based on the above assumption.
As shown above, the coefficients of the interpolation matrix are ΔR _i , Δ
It is determined by the covariance of G _i and ΔB _i . Generally, statistics such as covariance differ depending on the type of image such as a photograph or a line drawing. Therefore, by changing the interpolation matrix depending on the type of image, it is possible to perform interpolation processing suitable for the characteristics of the image. In this method, the probability density is maximized for the signal of the difference between adjacent pixels. Under such a condition, the DC component of the other color signal (G, B) and the difference (difference) are added to the estimation of a certain color signal (for example, R). The estimation error in reading is zero in principle.

In the embodiment of FIG. 3, which will be described later, the condition for adding this difference is taken into the circuit and the circuit and configuration are simplified.

Further, in the above-mentioned model, noise in a signal read by the sensor is not considered. However, in practice, the effect of noise may not be negligible. In such a case,
Optimal estimation can be performed by the same matrix operation as described above. However, in this case, the first row of the interpolation matrix A is not always (000 1000).

Next, a second embodiment used in the present invention will be described.

In this embodiment, interpolation is performed by the calculation shown in (19).

(However, the number at the right shoulder of the matrix A represents a suffix.) In the above-described first embodiment, A ⁰ = A ¹ = A ² ,
In the present embodiment, by providing these interpolation matrices A ⁰ , A ¹ , and A ² independently, optimal estimation can be performed even when the covariance between R, G, and B is not symmetric. As a result, the designation accuracy is slightly improved compared to the first embodiment, and higher resolution can be obtained.

FIG. 6 shows a configuration example of the extended interpolation circuit in this embodiment. This circuit has almost the same configuration as the extended interpolation circuit in the first embodiment, but the circuit scale is slightly larger than that in the first embodiment for switching the multiplication coefficient to nine.

Next, a third embodiment using the present invention will be described. In this embodiment, the arithmetic expressions are simplified on the assumption that the interpolation matrix A satisfies the condition of the expression (16) which further restricts the above-mentioned expression, and the operations shown in the expressions (20) and (22) are performed.

This operation is long is satisfied (16) _{conditions, A '10 = A 1,} -3, A' 11 = A 1, -2, A '12 =
^{_{A 1, -1 -1, A '}} 13 = A 1,0 A' 20 = A 2, -3, A '21 = A 2, -2 -1, A' 22 =
By _setting A _{2, −1} , A ′ ₂₃ = A _2,0 , the arithmetic expression (2 expressions) in the first embodiment is obtained.
Is equivalent to Under the conditions of Equation 16, when a uniform color is input, the estimation error becomes 0, so that the restriction of this condition poses no practical problem at all.

FIG. 7 shows a specific configuration example of the extended interpolation circuit of this embodiment. In this circuit, an input image signal is first input to a four-stage shift register 701. Then, the subtracter 702 subtracts the output signal of the third stage of the shift register 701 from the input signal to obtain a difference signal. This signal represents a difference between signals of the same color component separated by three pixels. This difference signal is further input to the second four-stage shift register 703. The multiplication circuit 704 multiplies the output signals of the first to four stages of this shift register by coefficients A ′ _jk (J = 1, 2) (K = 0 to 3).
The first shift register as shown in the figure.
The adder 705 adds the output signals of the third and second stages of 701 to the output signals. The signals Y _i , Z _i resulting from this addition and the output signal X _i at the fourth stage of the first shift register 701 are switched by the switch 706, whereby the interpolation output signals R ′ _i , G ′ _i , B ′ _i
Get. Here, the operation of this switch complies with equation (18).

As a result, the interpolation calculation represented by the equations (17) and (18) can be realized.

In this embodiment, the amount of calculation per pixel is greatly reduced to 8 multiplications and 8 additions, compared to the 21 multiplications and 18 solutions in the first embodiment, and the circuit scale can be reduced. .

Next, a fourth embodiment using the present invention will be described. In this embodiment, as shown in FIG. 8, a 1/2 reduction circuit 802 is provided after the extension interpolation circuit 801 in the first embodiment. In this circuit, the adder 804 adds and averages two consecutive RGB signals of two pixels and outputs a signal of one pixel. As a result, a reduction of 行 わ in the main scanning direction is performed, and a signal having a density of の of the element density of the line sensor is obtained.

This is compared to the conventional method of reading three elements as one pixel.
A signal density of 1.5 times is obtained, and a higher resolution than a signal obtained by enlarging the signal obtained by the conventional method by 1.5 times by simple interpolation can be obtained.

As described above, the present invention is effective even when combined with an enlargement / reduction circuit. A higher resolution can be obtained than when the same signal density is obtained by combining the enlargement / reduction circuit with the conventional method. In the above embodiments, the primary color RGB color filter is used as the color filter of the color line sensor. However, the present invention is not limited to this, and a complementary color filter such as CGY may be used. Further, in the above embodiment, a signal of a total of seven pixels before and after the pixel is used to perform the interpolation calculation of the pixel. However, the signal is not limited to seven pixels. For example, five, six, or eight or more pixels are used. May be.

[Effects of the Invention] According to the present invention, when three color signals at the positions of the respective light receiving elements are obtained by interpolation from signals read by the color sensor, interpolation is also performed using a signal of a color component different from the color signal to be obtained. Thus, a color signal with high resolution can be obtained.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of one embodiment of the present invention, FIG. 2 is an explanatory diagram showing the principle of interpolation in the present invention, and FIG. 3 shows the configuration of a sensor used in this embodiment in a dot-sequential manner. FIG. 4 is a diagram showing a configuration of an extended interpolation circuit in the first embodiment;
FIG. 5 is a diagram showing a configuration of an image signal input to the extended interpolation circuit, FIG. 6 is a diagram showing a configuration of the extended interpolation circuit in the second embodiment, and FIG. 7 is a extended interpolation in the third embodiment. FIG. 8 is a diagram showing a configuration of a circuit, and FIG. 8 is a diagram showing a configuration of a reduction circuit in the fourth embodiment. 201: Light source 202: Original surface 203: Signal reading area 204: Lens array 205: Color sensor 206: Carriage 207: Amplifier 208: A / D converter 209: Standardization circuit 210: Extended interpolation circuit 251, 252, 253 ... Image signal 401 ... Shift register 402 ... Multiplication circuit 403 ... Adder 404 ... Register 601 ... Shift register 602 ... Multiplication circuit 603 ... Adder 701 ... Shift Register 702—Subtractor 703—Shift register 704—Multiplier 705—Adder 706—Switcher 711-713—Interpolation signal 801—Expansion interpolation circuit 802—1 / 2 Reduction circuit 803 Register 804 …… Adder 805 …… Register

 ────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-61-154357 (JP, A) JP-A-61-214689 (JP, A) JP-A-61-227492 (JP, A)

Claims (7)

(57) [Claims] 1. A color sensor according to claim 1, wherein said color filters are read using a color sensor having a plurality of types of color filters arranged on said plurality of light receiving elements and have color components corresponding to any of said color filters. What is claimed is: 1. A color image processing method for obtaining a color signal other than a component corresponding to each light receiving element, wherein when obtaining a color signal other than a color component of the output signal of a light receiving element, the color signal of another light receiving element is obtained. A second output signal having a color component other than the color component of the color signal of at least one of the light receiving elements in addition to the first output signal having the color component
A color signal other than a color component included in the output signal of the light receiving element by performing a linear calculation using the first and second output signals. . 2. The color image processing method according to claim 1, wherein said linear calculation is performed by calculating a weighted average of said first and second output signals. 3. The first output signal according to claim 1, wherein said second output signal is an output signal of said certain light receiving element.
Item. 4. A color image processing method according to claim 1, wherein said plurality of types of color filters are arranged at the same density. 5. A color image processing method according to claim 1, wherein said color sensor is configured by sequentially arranging three types of color filters. 6. A color signal other than a color component of an output signal of the light receiving element by adding a difference of the second output signal to a linear sum of the first output signal. 2. The color image processing method according to claim 1, wherein 7. The color image processing method according to claim 1, wherein after obtaining the color signal, interpolation or thinning is performed between the same color components.