JP2006166191A - Speedup system of color conversion in multiple primary colors display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress an increase in time for color conversion based on a linear programming of a multiple primary colors display device and to reduce it almost equal to a time for color conversion of a simple matrix method without increasing power consumption. <P>SOLUTION: The speedup system of color conversion in a multiple primary colors display device is provided with: a first processing means for converting XYZ image data according to the linear programming, establishing a classification model by using its result with non-base (<SB>n</SB>C<SB>n-3</SB>*2<SP>n-3</SP>kind) out of a base/non-base combination as learning data, and generating a 3*3 inverse matrix of a corresponding base part as a 3*3 color conversion matrix; and a second processing means for obtaining a combination of optimal non-base from the inputted XYZ image by using the established classification model and the inverse matrix and performing calculation by a matrix corresponding to the combination. Thus, the system performs color conversion at high speed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a color conversion speed-up method used in a multi-primary color display device that shortens color conversion time without increasing power consumption in a display device that improves color reproduction.

In recent years, there are several references on high fidelity color reproduction. That is, the first non-patent document entitled “Spectrum-based high-color reproduction video system-natural vision” by Masahiro Yamaguchi, Hideaki Haneishi and Nagaaki Oyama (Technical Report of the Institute of Image Information and Television Engineers, Vol. 26, No. 58, 7-12) Page, 2002), and the second non-patent document titled “Development of a high-detail color management system based on human perception-its application to the reproduction of records of museums and museum collections” (1996, 1997 IPA original) According to the Information Technology Development Business Results Report), the importance of high fidelity color reproduction in the fields of electronic commerce, digital archives, and telemedicine is pointed out.
Masahiro Yamaguchi, Hideaki Haneishi, Nagaaki Oyama “Spectrum-Based High-Color Reproduction Image System-Natural Vision” IEICE Technical Report, 26, 58, 7-12, 2002 "Development of a high-detail color management system based on human perception-its application to the reproduction of records of museums and museum collections", 1996/1997 IPA creative information technology training business results report

  On the other hand, one of the causes of deterioration in color reproduction accuracy is a narrow color gamut of the display device. In the uniform chromaticity diagram (UCS), the color gamut of HDTV is narrower than the human visible region. Therefore, it is clear that the color range that can be displayed by HDTV is narrower than the visible region and the display region is not sufficient.

  Currently, research is being conducted on display devices with a wider color gamut than HDTV. The wide color gamut display device here is a display device that not only has a wide reproduction range on the chromaticity diagram, but also can reproduce a color of a wide color gamut without reducing the maximum luminance. In order to develop a wide color gamut display device with three primary colors like existing devices, primary colors with high saturation and high brightness are required. In general, the luminous efficiency of the light emitting device decreases as the saturation increases. For this reason, a wide color gamut display device using the three primary colors becomes inefficient. This can be solved by using primary colors that satisfy the following two conditions. That is, a primary color with high saturation but not so high brightness and a primary color with high brightness but not so high saturation are required. However, in this case, four or more primary colors are required.

  As an example of such a multi-primary color display device, there is a six-primary-color rear projection display device manufactured by Olympus, which is one of wide color gamut displays using multi-primary colors. By using four or more primary colors, a wide color gamut display device that is more efficient than the case of the three primary colors can be realized. However, new problems also arise as the primary colors increase. One of them is the degree of freedom of color conversion. In the existing three primary color display devices, since the color tristimulus values XYZ and the three primary colors RGB have a three-to-three relationship, there is no degree of freedom in conversion, and unique conversion is possible. However, as the number of primary colors increases, the tristimulus values and primary colors have a relationship of 3 to 4 or more, and therefore, there is a degree of freedom in converting the signal values of each primary color from the tristimulus values, and the conversion cannot be performed uniquely. This indicates that there are a plurality of combinations of primary colors for displaying a certain XYZ. For this reason, there are also combinations of primary colors with low luminous efficiency.

On the other hand, in the n primary color display device, the tristimulus values X, Y, and Z of the output color and the tristimulus values {(X ₁ , Y ₁ , Z ₁ ), (X ₂ , Y ₂ , Z ₂ ) of each primary color are output. _{), ···, (X n,} Y n, Z n)}
Have the relationship described in the following formula (1). For example, Non-Patent Document 3 can be referred to.
“International Color Consortium:“ ICC Profile Specification Version 3.2 ”” 1995 (“International Color Consortium,“ ICC Profile Specification, Version 3.2 ”” 1995)

（１）

(1)

  On the other hand, the relationship described in the following equation (2) is assumed between the tristimulus values of a certain primary color n and the corresponding input signal values.

（２）

(2)

Here, X _n , Y _n , and Z _n are tristimulus values of the primary color n when the maximum luminance is output, and _Sn is an input signal value for the primary color n. S _n is 0 ≦ S for taking the value of _n ≦ 1, it can be considered as normalized value of the maximum luminance in primary color n. The S _n will be referred to as relative brightness of the primary colors n later. When Expression (1) and Expression (2) are expressed using a summary matrix, conversion of the tristimulus values XYZ from the relative luminance values of the respective primary colors is given by the following Expression (3).

（３）

(3)

  Further, the conversion from XYZ to the relative luminance of each primary color is expressed by the following expression (4) from the expression (3).

（４）

(4)

  Here, in Equation (4), when the number of primary colors is 3, the coefficient matrix is a square matrix, so that the inverse matrix is uniquely determined, and conversion can be performed without any problem. However, when the number of primary colors is 4 or more, the matrix becomes a coefficient of 3 * n (n> 3), so that an inverse matrix cannot be obtained uniquely.

  Several studies have been conducted to deal with this problem and published as fourth to sixth non-patent documents.

The fourth non-patent document is a paper entitled “Color Matching Experiment Using Six Primary Color Displays” written by Takeshi Terachi, Kenro Osawa, Masahiro Yamaguchi, and Nagaaki Oyama (Color Forum JAPAN 2001, pp. 97-100, 2001).
This document proposes a method of using different color matching functions for each observer without using CIE-XYZ color matching functions for color reproduction, and a method of reproducing the shape of spectral radiance estimated by a multispectral camera. . Color conversion can be uniquely performed by matching the number of dimensions of the color matching function with the number of dimensions of the display device or by reproducing the spectral radiance with no degree of freedom. However, it is not realistic to measure the color matching function for each individual or to use a multispectral camera.
Takeshi Terachi, Kenro Osawa, Masahiro Yamaguchi, Nagaaki Oyama "Color Matching Experiments Using Six Primary Color Displays" Color Forum JAPAN 2001, 97-100 pages 2001

The fifth non-patent literature is Takeyuki AJITO, Kenro OHSAWA, Takashi OBI, Masahiro YAMAGUCHI
and "Color Conversion Method for Multiprimary Display" by Nagaaki OHYAMA
Article titled “Using Matrix Switching” (Optical Review, Vol.8, No.3, pp.191-197 (2001)) by Yuki Ajito, Kenro Osawa, Takashi Obita, Masahiro Yamaguchi, and Nagaaki Color conversion method of used multi-primary color display device "Optical Review, Vol. 8, No. 3, 191-197 (2001)).
This document proposes a method of performing color conversion uniquely by dividing a color gamut created by primary colors and assigning priorities. This method has an advantage that high-speed conversion can be performed, but the degree of freedom is eliminated only by the priority order for each region, and other effective usage methods of the degree of freedom cannot be considered.
Takeyuki AJITO, Kenro OHSAWA, Takashi OBI, Masahiro YAMAGUCHIand Nagaaki OHYAMA "Color Conversion Method for Multiprimary Display Using Matrix Switching" Optical Review, Vol.8, No.3, pp.191-197 (2001). Obitakashi, Yamaguchi Masahiro, Oyama Yamaaki, “Color Conversion Method for Multi-Primary Color Display Using Matrix Switch”, Optical Review, Vol. 8, No. 3, 191-197 (2001))

The sixth non-patent document is a paper titled “Color conversion for a multi-primary display using linear interpolation on equi-luminance plane method (LIQUID)” by Hideto Motomura (Journal of the SID, 11/2, pp.371 -387 (2003)) ("Color conversion of multi-primary color display device using linear interpolation on isoluminous surface" by Motomrahideo, SID Journal, Vol. 11, No. 2, pages 371-387 (2003)). is there.
In this document, conversion with no degree of freedom can be performed by using a linear interpolation method using three points with equal luminance. However, as in the fifth non-patent document, it is not possible to consider other effective usage methods besides the degree of freedom.
Hideto Motomura "Color conversion for a multi-primary display using linear interpolation on equi-luminance plane method (LIQUID)" Journal of the SID, 11/2, pp.371-387 (2003) (Motomura Hideo Color conversion of multi-primary color display device using linear interpolation method ”SID Journal, Vol. 11, No. 2, pages 371-387 (2003))

The linear programming problem is an optimization problem that maximizes or minimizes the objective function z under the constraint conditions, and the constraint conditions and the objective function are expressed in a linear form. The 7th non-patent literature, Masatoshi Sakawa, “Optimization of Linear System <From One Purpose to Multipurpose>”, Morikita Publishing (1984), is a reference for this problem. Since the problem can be treated algebraically under the condition of linearity, an optimal solution can be obtained simply as compared with the nonlinear optimization problem. The standard form of the linear programming problem is shown in Equation (5).
Masatoshi Sakawa, “Optimization of Linear Systems (From One Purpose to Multiple Purposes)” Morikita Publishing 1984

Minimization z = c ₁ x ₁ + c ₂ x ₂ +... + C _n x _n
Objective function a ₁₁ x ₁ + a ₁₂ x ₂ +... + A _1n x _n ± d ₁ = b ₁
a ₂₁ x ₁ + a ₂₂ x ₂ +... + a _2n x _n ± d ₂ = b ₂
・ ・ ・ ・ ・ ・
a _m1 x ₁ + a _m2 x ₂ +... + a _mn x _n ± d _m = b _m
0 ≦ x _j , d _i (j = 1, 2,..., N)
a _ji , c _j : constant
(J = 1, 2,..., N: i = 1, 2,..., M)
(5)

On the other hand, the following two basic theorems are given to linear programming. That is,
The first is "if there is a feasible solution, there is always a feasible base solution",
The second is that “if there is an optimal solution, there is an optimal solution among feasible base solutions”.

  By using this theorem, an optimal solution can be obtained by a finite number of combinatorial searches. However, as the number of variables increases, the number of combinations increases and processing takes time. Therefore, the simplex method is used. In the simplex method, the optimal solution can be obtained by efficiently using the basic theorem of linear programming by judging the optimality using the relative cost coefficient.

  Next, linear programming in color conversion will be described. In order to use linear programming for color conversion, a conversion function from the relative luminance value of each primary color to XYZ and the range that the relative luminance value can take are set as constraints, and any objective function may be set. The following formula (6) shows a color conversion problem that has been converted to the standard form of linear programming.

Minimization z = c ₁ S ₁ + c ₂ S ₂ +... + C _{n Sn}
Objective function X ₁ S ₁ + X ₂ S ₂ +... + X _n S _n = X
_{Y 1 S 1 + Y 2 S} 2 + ··· + Y n S n = Y
_{Z 1 S 1 + Z 2 S} 2 + ··· + Z n S n = Z
0 ≦ S _j ≦ 1 (j = 1, 2,..., N)
(6)

Comparing equations (5) and (6), there are two differences compared to the normal linear programming problem. One is the range that the variable can take. In the normal linear programming problem, the range that the variable can take is 0 ≦ x _n , but the range of relative luminance is 0 ≦ S _n ≦ 1 when performing color conversion, and the simplex method is applied as it is. I can't. This can be dealt with by introducing an insufficient variable for S _n ≦ 1 and solving it by incorporating it into the constraint condition as S _n + α = 1, or by using the upper limit method. Reference books on this problem include the eighth non-patent document, “Upper bounds, secondary constraints and block triangularity in linear programming” by GB Dantzig, Econometrica, 23, pp.174-183 (1955). Danzig, “Upper limit, second limit, and block triangles in linear programming”, Econometrica, 23, 174-183, 1955).
GB Dantzig “Upper bounds, secondary constraints and block triangularity in linear programming” Econometrica, 23, pp.174-183 (1955) , 23, 174-183, 1955)

Another difference is that there is no lack / margin variable (d _j ) in the color conversion problem. In normal linear programming, the initial feasible basis solution is obtained by setting the deficit / margin variable as a non-basis variable (= 0), and the simplex method is advanced using that. In the linear programming used for color conversion, there is no shortage / margin variable and the upper limit value is given to the variable. Therefore, various combinations of two types of non-basis variables, 0 and 1, must be sought. It becomes difficult to find an initial feasible basis solution. For this purpose, it is necessary to use a two-stage simplex method. The two-stage simplex method finds an initial feasible basis solution in the first stage or obtains information that it does not exist. In the second stage, an optimal solution is found from the initial feasible basis solution, or information is obtained that the solution is not bounded (there is a solution that is too small).

Next, color conversion considering power consumption will be described. Since the constraint conditions have been determined as described above, color conversion can be performed only by giving an objective function. Since there is a degree of freedom in multi-primary color conversion, there are also combinations of primary colors that increase power consumption in self-luminous display devices. To determine the value of the primary color to minimize power consumption by using a linear programming method, it must represent the relative luminance S _n primaries n power consumption P _n in the linear formula such as the following equation (7) . In addition, the power consumption of the multi-primary color display device that satisfies Expression (7) is Expression (8).

P _n = c _n × S _n (7)

P ₁ + P ₂ +... + P _n = c ₁ S ₁ + c ₂ S ₂ +... + C _{n Sn} (8)

  The above equation (8) matches the objective function z in equation (6). There is a time gray scale control as one method of the gray scale control in which the power consumption is actually in the linear form as in the equation (8). Although other control methods are also non-linear, they have a feature that the luminance increases as the power consumption increases. The basis solution that can be obtained even if the accuracy of the objective function is not so accurate in linear programming satisfies the constraints. For this reason, even if the relationship between the relative luminance and the power consumption is approximated by the equation (7), there is no problem.

  The above linear programming method can reduce the power consumption of the display device, but the processing time required for color conversion is long in a large display device. For example, in a SHIPP XYZ image, when color conversion is performed by linear programming, the processing time for one SHIPP image is 3 minutes and 10 seconds. On the other hand, in the case of three primary colors, the color conversion time is 14 seconds at the maximum calculation amount. As described above, the color conversion time is significantly increased in the linear programming method as compared with the simple matrix method.

  In the above color conversion method for such a multi-primary color display device, the purpose of color conversion can be achieved without increasing the power consumption of the display device by using linear programming, but the color conversion time is significantly increased. There are challenges. By solving this by introducing a decision tree, it is required to shorten the color conversion time without increasing the power consumption of the display device.

  In order to solve the above-described problems, the present invention regards color conversion by linear programming as a discrete classification problem, and performs color conversion processing by classification and simple matrix calculation by learning the color conversion result by linear programming using a classification model. To provide a color conversion method capable of shortening the color conversion time.

  The result of color conversion by linear programming is learned by induction into a classification model, and classification and simple matrix calculation are performed, so that, for example, a 16 × 3 bit SHIPP image having a resolution of 4,096 × 3,072 pixels For (yachts), in the case of displaying six primary colors, the color conversion time is reduced from 320 seconds to 21 seconds. The color conversion time of the present invention adopting the classification model is about 10 times faster than the color conversion time of the linear programming method. This is almost the same as the color conversion time of the simple matrix method is 18 seconds.

  When multi-primary color conversion is performed using linear programming, there is a problem that the conversion speed becomes slow. In order to perform color conversion without impairing the advantages of color conversion of linear programming, a method of inductively learning the result of color conversion by linear programming is conceivable. In this case, it is the simplest method to construct a model with the input values as XYZ tristimulus values and the output values as multi-primary color signals. However, it is difficult to build a model for accurately outputting multi-primary color signals that are continuous values. Therefore, in the present invention, the result of color conversion by linear programming is treated as a discrete value, and these are subjected to inductive learning in a classification model to perform color conversion.

In order to achieve high speed while guaranteeing high fidelity color reproduction, it is only necessary to use a combination of a basis / non-basis of linear programming instead of a model that directly outputs color signals. In the color conversion method using the linear programming method, (the number of primary colors −3) among the obtained solutions is the highest (tone: 255) or no light (tone: 0). These extreme solutions are called non-basic solutions. When the optimal non-basic combination ( _n C _n-3 * 2 ^n-3 ) is determined, the optimal output of the linear programming is obtained as an output value of the classification model. It can be handled as a three-primary color system using a 3 * 3 conversion matrix that matches the values. According to this method, it is possible to perform multi-primary color conversion at high speed while maintaining the fidelity without detracting from the advantages of linear programming simply by performing matrix operations.

  The result of the above linear programming is learned with a classification model. This time, a decision tree is adopted as a model for classifying. Since decision trees are by no means a fast classification model, it is necessary to consider introducing other models in order to achieve higher speed. It is also appropriate to use rule-based C4.5 as a decision tree implementation. Since the rule is given as X '<X <X ", it is considered that the rule-based C4.5 performs area division by a rectangular parallelepiped in the XYZ space.

  Summarizing the above, the color conversion method of the multi-primary color display device according to the present invention includes, as shown in FIG. 1, first processing means for learning a color conversion result by linear programming into a classification model, a classification model, and a matrix calculation. And a second processing means for performing color conversion based on the above, and configured to reduce the processing load of the processing means based on the linear programming.

The processing part for causing the classification model to learn the color conversion result by the linear programming method is that the optimal solution among the non-basic solutions given by the combination of the basis / non-basis of the linear programming method is the non-basic ( _n C _n-3 * 2 ^n-3 ), a learning model is recursively learned by the relationship of many XYZ vs. non-basis combinations, and the output value is set to 3 using a 3 * 3 color conversion matrix adapted to the output value. It is for handling as a primary color.

  The color conversion processing means based on the classification model and the matrix calculation outputs an optimal combination of non-basis solutions from the input XYZ by the constructed classification model, and generates a 3 * 3 color conversion matrix suitable for the output value. To perform color conversion.

In order to confirm the above-described invention in detail, a specific calculation process is executed for an actual image, and the color conversion processing time is obtained. It is important to construct the model and classify by model. A processing flow of the present invention is shown in FIG. In FIG. 2, all the correspondences between the input (X _C , Y _C , Z _C ) and the total number M are obtained in advance by linear programming. That is, the data X _C , Y _C and Z _C are input to the decision tree, and the input data is classified into M combinations. Based on this result, a basis solution (S ₁ , S ₂ , S ₃ ) and a non-basis solution (M combinations of S ₄ , S ₅ , S ₆ ) are obtained, and further color conversion is performed by inverse transformation and matrix calculation. Do. Thereby, the gradation value as a digital signal is obtained based on the tone curve of the target display.

In the multi-primary color (n primary color) color conversion, as described in the background art above, the number of equations is three and the number of variables is n. Therefore, the number of variables is n, the number of expressions is 3, and the degree of freedom is n-3. From the above, there are L = _n C _n−3 2 ⁿ⁻³ combinations of non-basis (0 and upper limit value 1). Solving linear programming to find the optimal solution is equivalent to choosing an optimal combination from L ways.

This is clearly a discrete classification problem. There are various ways to solve the classification problem. Color conversion is calculated in advance by linear programming from an image taken by a camera capable of acquiring SHIPP or XYZ. It is necessary to train the model using the results calculated in advance as training data.
The procedure for performing color conversion by such a method is as follows. First, if possible, L inverse matrices are preloaded into the memory. Second, by inputting XYZ into the constructed model, one of L combinations is obtained. Third, color conversion is performed using an inverse matrix corresponding to the combination.

  The above method has two problems. The first is to make a mistake in the solution, that is, to output a wrong combination from the above-mentioned L ways, and the second is that the solution cannot be obtained. The first problem may include a problem that an optimal solution cannot be obtained, but there is no particular problem in terms of color conversion. Color conversion is possible if a solution is obtained for any of the L combinations. The second problem can also be solved by passing to linear programming when a solution cannot be obtained.

Therefore, we learned the result of color conversion using a decision tree, which is one of the classification methods. The decision tree is used for the following three reasons.
First, the output class can handle discrete values. Second, classification can be performed in a linear domain division. Third, as a learning result, it is possible to output a rule for classification as a sentence, so that it is easy to incorporate it into a program.

  The decision tree package used is C4.5. The number of learning data was 11102. As a result of learning with the default option, the number of rules was 198. An example of the output result is shown in FIG.

  In FIG. 3, the values of X, Y, and Z are multiplied by 100 to form an integer type in order to discard fine digits. In the output class, * is a base, 1 and 0 are non-bases, and are Lr, Lg, Lb, Rr, Rg, and Rb from the left. Here, Lr, Lg, and Lb are the color signals (relative luminance) of the primary colors r, g, and b on the short wavelength side, and Rr, Rg, and Rb are the primary colors of r, g, and b on the long wavelength side, respectively. It is a color signal.

  Since it is a linear programming problem, a linear optimal solution space should be constructed, but as a result, 198 conditions were output. On the other hand, since the noise data is included in the dark portion in the image used for learning, 156 rules were generated as a result of learning the above result by changing the minimum appearance frequency condition option.

  Since the speeding up of the color conversion process has been described, the color conversion calculation time is easily compared between the linear programming method and the classification method. In the linear programming method, when XYZ values are input, multiprimary color signal values are output. On the other hand, when the classification method is used, the output when XYZ is input is a combination pattern of a base and a non-base. In order to obtain the signal values of multi-primary colors, it is further necessary to obtain an inverse matrix from the obtained pattern and perform matrix calculation using it. Since the number of inverse matrices used is limited, they should be prepared in advance. Since only confirmation of speedup was performed, the matrix and the like were generated by random numbers, and the calculation load was made equivalent to color conversion.

  The number of pixels corresponding to the number of pixels of an image captured by an imaging device capable of acquiring XYZ, that is, X, Y, Z calculated randomly at 1392 * 1040 times is used to calculate the color signal value by linear programming and classification. Calculated. The calculation time in the linear programming method was 42.5 seconds. On the other hand, the calculation time by the classification method was 4.5 seconds. It can be said that the speed can be increased by using the classification method. In another example, for a SHIPP image (yacht) consisting of 16 × 3 bits having a resolution of 4,096 × 3,072 pixels, for example, in the case of displaying six primary colors, the color conversion time is 320 seconds to 21 seconds. Shortened to

  The color conversion method for the multi-primary color display device according to the present invention can be widely used for the color reproduction optimization design of the multi-primary color display device and the efficiency design of the display device.

It is an example of the color conversion system of a multi-primary color display device. It is explanatory drawing which shows the example of calculation of the color conversion based on a decision tree. It is an example of an output showing the result of color conversion using a decision tree.

Explanation of symbols

  nothing special

Claims (2)

The XYZ image data is transformed by linear programming, and the classification model is constructed by using this result and non-basis ( _n C _n-3 * 2 ^n-3 types) of the basis / non-basis combinations as learning data. First processing means for generating a 3 * 3 inverse matrix of a corresponding base portion as a 3 * 3 color conversion matrix;
Using the constructed classification model and the inverse matrix, a second processing means for obtaining an optimal non-basis combination from the input XYZ image data and performing calculation using the corresponding matrix, and performing color conversion at high speed Speed-up method of color conversion in a multi-primary color display device configured to be able to perform the above. Improving the accuracy of color conversion by color conversion processing by linear programming in the first processing means;
2. The method for speeding up color conversion in a multi-primary color display device according to claim 1, wherein the second processing means is configured to reduce a load of calculation processing on XYZ image data.

Images