JP6813749B1 - How to quantify the color of an object, signal processor, and imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and an apparatus for appropriately evaluating the color of an object whose surface color is not uniform. A method of quantifying the color of an object is as follows: S101, each of which acquires one or more pixel values corresponding to a plurality of colors, extracts n pixel blocks from an image, and S102, each pixel block Each representative value of one or more colors is determined S103, for each pixel block, the representative value of each pixel value of one or more colors and all other pixel blocks on the second color space. Calculate the sum of the values indicating the degree of difference from the representative value of each pixel value of one or more colors of S105, from the n pixel blocks having the smallest sum of the values indicating the degree of difference. S107 determines and outputs a value indicating one color based on a representative value of each pixel value of one or more colors in m pixel blocks (m is an integer of 2 or more and less than n) selected in order. [Selection diagram] Fig. 4

Description

The present disclosure relates to a method for quantifying the color of an object, a signal processing device, and an imaging system.

Conventionally, various techniques for measuring or evaluating color have been developed. For example, Patent Documents 1 to 3 disclose examples of techniques for measuring or evaluating the color of an object.

Japanese Unexamined Patent Publication No. 2016-20760 Japanese Unexamined Patent Publication No. 2014-132257 Japanese Unexamined Patent Publication No. 2009-25867

The present disclosure provides novel methods and devices for appropriately evaluating the color of an object whose surface color is not uniform.

The method of quantifying the color of an object according to one aspect of the present disclosure includes a step of acquiring image data including a plurality of pixels each having one or more pixel values corresponding to one or more colors. From the image, the step of extracting n pixel blocks (n is an integer of 3 or more) and the representative value of each pixel value of the one or more colors for each of the extracted n pixel blocks are set. The step of determining, the representative value of each pixel value of the one or more colors for each of the n pixel blocks, and the 1 for all other pixel blocks included in the n pixel blocks. From the step of calculating the sum of the values indicating the degree of difference between the pixel values of each of the two or more colors and the representative value, and from the n pixel blocks having the smallest sum of the values indicating the degree of difference. A step of determining and outputting a value indicating one color based on a representative value of each pixel value of the one or more colors in m pixel blocks (m is an integer of 2 or more and less than n) selected in order. including.

The above-mentioned comprehensive aspect may be realized by an apparatus, a system, an integrated circuit, a computer program, a recording medium, or any combination thereof.

According to the embodiment of the present disclosure, it is possible to appropriately evaluate the color even for an object whose surface color is not uniform.

It is a figure which shows typically the structure of the imaging system in an exemplary embodiment of this invention. It is a figure which shows an example of the structure which provided the reflection type lighting apparatus. It is a flowchart which shows the outline of the processing executed by a signal processing apparatus. It is a flowchart which shows the flow of the color registration process in step S10. It is a figure for demonstrating the example of the extraction process of a pixel block. It is a figure for demonstrating the example of the extraction process of a pixel block. It is a figure for demonstrating the calculation of the sum total of the distances of a plurality of points in L ^* a ^* b ^* color space. L ^* a ^* b ^* It is a figure for demonstrating the calculation of the sum total of the distances of a plurality of points in a color space. It is a figure which shows the example of m points which the sum of the distances from n points in L ^* a ^* b ^* color space to other points is relatively small. It is a flowchart which shows the flow of a color determination process. It is a figure which shows the example of the method of the quality determination. It is a conceptual diagram which shows the example of the quality judgment based on statistical data. It is a figure which shows the Macbeth color checker. It is a figure which shows the effect by the coefficient adjustment method in an exemplary embodiment.

Conventionally, a method such as color measurement using a colorimeter or spectroscopic analysis using a spectrum measuring device has been used for color measurement and evaluation. These methods generally measure color in small spots (eg, a few millimeters or less). Therefore, it is difficult to measure the color of an object whose surface color is not uniform with high accuracy. For example, it is difficult to measure the color of fabrics with tints, creases, etc., paper (especially those with large fibers), rugs, walls, or uneven hand-painted paints or inks with high accuracy. When measuring such an object, the measured values differ greatly even if the positions of the measurement points are slightly different. Therefore, it has not been possible to obtain stable measurement results for the above-mentioned objects by the conventional color measurement method.

Therefore, the present inventors have developed the methods and devices of the respective embodiments described below in order to appropriately quantify and evaluate the color of an object whose surface color is not uniform. It was. The outline of the embodiment of the present disclosure will be described below.

The method of quantifying the color of an object according to an embodiment of the present disclosure includes the following steps (A) to (E).
(A) Acquire image data including a plurality of pixels each having one or more pixel values corresponding to one or more colors.
(B) N pixel blocks (n is an integer of 3 or more) are extracted from the image.
(C) For each of the extracted n pixel blocks, a representative value of each pixel value of the one or more colors is determined.
(D) For each of the n pixel blocks, the representative value of each pixel value of the one or more colors and the one or more for all the other pixel blocks included in the n pixel blocks. Calculate the sum of the values indicating the degree of difference from the representative value of each pixel value of the color.
(E) Among the n pixel blocks, one or more of the m pixel blocks (m is an integer of 2 or more and less than n) selected in order from the one having the smallest sum of values indicating the degree of the difference. A value indicating one color based on a representative value of each pixel value of the color is determined and output.

The "value indicating the degree of difference" in the above step (D) may be the difference itself between the representative values of the pixel values of the one or more colors in the two pixel blocks to be compared, or the said one. It may be the difference between other values converted from the representative value of the pixel value. The "value indicating one color" in the above step (E) may be the average value of the representative values of the pixel values of the one or more colors in the m pixel blocks, or the representative. It may be another value obtained by a value-based calculation.

The value indicating one color in the above step (E) can be used as identification data for identifying the color of the object. The one or more colors may be a single color or a plurality of colors. When the one or more colors are monochromatic, the degree of shading of the object can be evaluated by one value by the above method. When the one or more colors are a plurality of colors, the color of the object can be evaluated by a plurality of values. The plurality of colors may include, for example, the three primary colors of red (R), green (B), and blue (B). However, it is not necessarily limited to the three primary colors. Depending on the application, the above method can also be used for evaluation of multispectral colors such as 4 colors or 5 colors. Here, although the expression "color" is used for convenience, the "color" in the present disclosure is not limited to the color included in the wavelength range of visible light, and also includes the wavelength range of invisible light such as infrared light or ultraviolet light. Including.

According to the above method, m pixel blocks estimated to have colors or densities close to each other are selected from the n pixel blocks extracted from the image. Then, a value indicating one color based on the representative value of each pixel value of the one or more colors of the m pixel blocks is output as color identification data. Such processing makes it possible to appropriately evaluate the color of an object whose color or density is not uniform.

When the one or more colors are a plurality of colors, the step (D) for calculating the sum of the values indicating the degree of the difference may include the following steps.
(D1) For each of the n pixel blocks, a plurality of representative values of the pixel values of the plurality of colors in a second color space different from the first color space composed of the plurality of colors. Convert to the value of.
(D2) For each of the n pixel blocks, the points on the second color space indicated by the plurality of values, and the plurality of points for all the other pixel blocks included in the n pixel blocks. The sum of the distances to the points on the second color space indicated by the value of is calculated as the sum of the values indicating the degree of the difference.

The value indicating one color may be a representative value of each of the plurality of values in the m pixel blocks selected in order from the n pixel blocks having the smallest total distance.

According to the above method, the color of the object can be evaluated more appropriately.

The method of quantifying the color of an object according to another embodiment of the present disclosure includes the following steps (a) to (f).
(A) Acquires RGB image data including a plurality of pixels, each having red (R), green (G), and blue (B) pixel values, generated by imaging an object.
(B) N pixel blocks (n is an integer of 3 or more) are extracted from the RGB image.
(C) For each of the extracted n pixel blocks, a representative value of each pixel value of R, G, and B is determined.
(D) For each of the n pixel blocks, the representative values of the respective pixel values of R, G, and B are different from the first color space composed of R, G, and B. Convert to three values in color space.
(E) For each of the n pixel blocks, the points on the second color space indicated by the three values, and the third for all the other pixel blocks included in the n pixel blocks. The sum of the distances to the points on the second color space indicated by the two values is calculated.
(F) Among the n pixel blocks, the representative values of each of the three values in the m pixel blocks (m is an integer of 2 or more and less than n) selected in order from the one having the smallest total distance. Calculate and output.

The representative values of each of the three values in the second color space in the above step (f) are used as identification data for identifying the color of the object. The representative value of each of the above three values may be, for example, the average value of each of the above three values in the m pixel block. According to the above method, m pixel blocks estimated to be close in color to each other are selected from the n pixel blocks extracted from the RGB image. Then, the representative values of each of the three values in the second color space of the m pixel blocks are output as color identification data. By such a process, it becomes possible to appropriately evaluate the color of an object whose color is not uniform.

The above method can be used, for example, to inspect the color of products produced by similar steps. In such an application, first, identification data of a pre-produced product (referred to as a first object) is recorded in advance on a recording medium as reference data by the above method. Next, identification data is generated for the second object produced by the same process as the above product by the same method as above. By comparing the generated identification data with the reference data recorded in advance, it is possible to determine whether the color of the object is good or bad.

In such applications, the above method may further include the following steps:
(G) A point on the second color space indicated by a representative value (for example, an average value) of each of the three values in the m pixel block, and a reference to each of the three values recorded in advance. The distance to the point on the second color space indicated by the value is calculated.
(H) The quality of the color of the object is determined according to the distance, and the determination result is output.

Whether the color of the second object is good or bad based on the degree of approximation between the color of the first object recorded in advance and the color of the second object to be inspected in steps (g) and (h). Can be determined. The quality of the color may be evaluated by, for example, a binary value of good / bad, or may be evaluated by three or more values depending on the degree of approximation.

When the above method is used when registering the color of an object, the step of calculating and outputting the representative value of each of the three values is to generate the representative value (for example, the average value) of each of the three values. It further includes recording on a recording medium as a reference value. When inspecting the color of other objects, the method further comprises the following steps:
(A') Acquire data of another RGB image generated by imaging another object.
(B') Another n pixel blocks are extracted from the other RGB image.
(C') For each of the extracted other n pixel blocks, a representative value of each pixel value of R, G, and B is determined.
(D') For each of the other n pixel blocks, the representative values of the respective pixel values of R, G, and B are converted into three values in the second color space.
(E') For each of the other n pixel blocks, the points on the second color space indicated by the three values and all the other pixels included in the other n pixel blocks. The sum of the distances to the points on the second color space indicated by the three values for the block is calculated.
(F') Among the other n pixel blocks, the representative value of each of the three values in the m pixel blocks selected in order from the one having the smallest sum of the distances is calculated.
(G') A point on the second color space indicated by a representative value of each of the three values in the other m pixel blocks, and a reference value of each of the three values recorded in advance. The distance to the point on the second color space indicated by is calculated.
(H') The quality of the color of the other object is determined according to the distance, and the determination result is output.

The three values in the second color space can be, for example, an L ^* value, an a ^* value, and a b ^* value in the L ^* a ^* b ^* color space. The L ^* a ^* b ^* color space is designed to approximate human vision. Therefore, by selecting in order from the pixel block having the smallest total distance in the L ^* a ^* b ^* color space, it is possible to quantify the value closer to human vision. It should be noted that the second color space may be a color space different from the L ^* a ^* b ^* color space because it does not necessarily have to be close to human vision depending on the purpose of quantifying the color.

The above method is a step of determining by machine learning the conversion coefficient used when converting the representative values of the respective pixel values of R, G, and B into the three values in the second color space. May further be included.

The step of determining the representative value of each of the R, G, and B pixel values may include a step of smoothing and averaging the respective pixel values of R, G, and B.

A signal processing unit according to another embodiment of the present disclosure includes a processor and a memory for storing a computer program executed by the processor. The computer program causes the processor to perform the above method.

An imaging system according to another embodiment of the present disclosure includes the above signal processing device and an imaging device that generates data of the RGB image by imaging the object.

Hereinafter, embodiments of the present disclosure will be described. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the inventors are intended to limit the subject matter described in the claims by those skilled in the art by providing the accompanying drawings and the following description in order to fully understand the present disclosure. is not. In the present specification, the same or similar components are designated by the same reference numerals. The shape and size of the whole or a part of the structure shown in the drawings of the present application do not limit the actual shape and size. In addition, other embodiments may be configured by appropriately combining the configurations of the embodiments described below.

(Embodiment)
[1. Constitution]
FIG. 1 is a diagram schematically showing a configuration of an imaging system 100 according to an exemplary embodiment of the present invention. The system 100 identifies the color of the object 230 and outputs the identification result. The object 230 is a non-uniform surface color, such as a fabric with tints or creases, paper (especially large fibers), rugs, walls, or uneven hand-painted paint or ink. is there. The object 230 is not limited to an object having a non-uniform color, and any object can be the object 230. The object 230 can be a product mass-produced, for example, by hand or on a production line. In that case, color variation may occur for each product. Therefore, the color identification process in the present embodiment is executed for each of the produced objects 230, and the quality of the color is determined. In the following description, as an example, it is assumed that the object 230 is a cloth having a tint block or a crease.

The system 100 includes a signal processing device 110, a lighting device 120, an image pickup device 140, a control device 160, and a display 220. The control device 160 is connected to the signal processing device 110, the lighting device 120, and the image pickup device 140. The display 220 is connected to the signal processing device 110.

The lighting device 120 in this example has a light emitting surface on which the object 230 can be placed, and emits visible light from the light emitting surface. The illuminator 120 can be realized, for example, by an array of a plurality of light emitting diodes (LEDs) that emit white light. The lighting device 120 may be realized not only by an LED but also by a light source such as a fluorescent lamp or an organic light emitting element. The light emitting surface functions as a table on which the object 230 is placed. The lighting device 120 emits white light from the back side of the object 230 toward the image pickup device 140. The white light emitted by the illuminating device 120 is not particularly limited, but for example, white light having a spectrum close to the spectrum of light from the D50 light source can be used.

The transmissive lighting device 120 as in the present embodiment can be used when the object 230 is a thin object such as cloth or paper. A reflective illuminating device may be provided instead of the transmissive illuminating device 120. The lighting device 120 is arranged as needed. If external lighting, natural light, or the like can be used, it is not necessary to provide the lighting device 120.

The image pickup apparatus 140 includes an image sensor having sensitivity in the wavelength range of visible light. In the example shown in FIG. 1, the image pickup apparatus 140 is arranged at a position where the light emitted from the illumination apparatus 120 and transmitted through the object 230 is received. The image pickup apparatus 140 takes an image of the object 230 and outputs an RGB image signal having information on each color of red (R), green (G), and blue (B). The image pickup apparatus 140 typically includes an image sensor in which three types of color filters, R, G, and B, are arranged for each pixel. The RGB image signal output by the image pickup apparatus 140 is data of a two-dimensional RGB image including a plurality of pixels in which each pixel has three pixel values of R, G, and B.

The control device 160 is a device that controls the signal processing device 110, the lighting device 120, and the image pickup device 140. The control device 160 controls the timing of light emission by the lighting device 120, imaging by the image pickup device 140, and signal processing by the signal processing device 110. The control device 160 is, for example, a general-purpose or dedicated computer.

The signal processing device 110 is a device that processes image data output from the image pickup device 140. The signal processing device 110 is, for example, a general-purpose or dedicated computer. The signal processing device 110 includes a processor (processing circuit) 112 such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), a recording medium such as a memory 114, and an input / output interface (IF) 116. The signal processing device 110 processes the image signal output from the image pickup device 140 to identify the color of the object 230, and outputs the identification result to the display 220. This operation is realized by the processor 112 executing a computer program stored in the memory 114.

The control device 160 and the signal processing device 110 may be realized by one computer. At least some of the functions of the signal processing device 110 may be performed by a computer installed far away from the imaging system 100. For example, a server computer connected to the control device 160 via a network such as the Internet may perform at least a part of the functions of the signal processing device 110.

The display 220 displays the color identification result according to the instruction from the signal processing device 110. The display 220 is a monitor such as a CRT, a liquid crystal, or an organic EL.

Instead of the configuration shown in FIG. 1, the configuration shown in FIG. 2 may be adopted. FIG. 2 is a diagram showing an example of a configuration in which the reflective lighting device 120 is provided. In this example, the lighting device 120 is arranged in the vicinity of the imaging device 140. The illuminating device 120 includes three light sources that emit light in the wavelength ranges of red (R), green (G), and blue (B), respectively. Each light source is, for example, an LED light source. In this example, the image pickup apparatus 140 may include a monochrome image sensor. That is, the image sensor does not need to be provided with R, G, and B color filters for each pixel. The control device 160 causes the lighting device 120 to sequentially turn on the light sources of R, G, and B, and causes the image pickup device 140 to acquire an image of each color. By synthesizing the three image signals of R, G, and B thus acquired, it is possible to generate color image data having pixel values of R, G, and B for each pixel.

[2. motion]
FIG. 3 is a flowchart showing an outline of the processing executed by the signal processing device 110. In the present embodiment, the color registration process of the object 230 is performed as a preliminary preparation for the process of determining the quality of the color of the object 230 (step S10). In this registration process, the signal processing device 110 determines a numerical value indicating the color of the object 230 by a process described later, and records the numerical value as a reference value in a recording medium such as a memory 114. After the registration process is completed, the color determination process (step S20) of the object 230 of the same type as the object 230 whose color is registered is performed. Specific examples of each process will be described.

[2-1. Color registration process]
FIG. 4 is a flowchart showing the flow of the color registration process in step S10. The processor 112 in the signal processing device 110 registers the color of the object 230 by executing steps S101 to S107.

Prior to step S101, the object 230 is photographed by the imaging device 140. Photography can be performed, for example, in a dark room to reduce the effects of ambient light. By this photographing, the RGB image data of the object 230 is output from the image pickup apparatus 140 and recorded in the memory 114 of the signal processing apparatus 110.

In step S101, the processor 112 acquires the RGB image data generated by the image pickup apparatus 140 from the memory 114.

In step S102, the processor 112 extracts a plurality of (n) pixel blocks from the acquired RGB image data. The number n of pixel blocks to be extracted can be set to any number of 3 or more. In some examples, n can be set to a number of about 100.

5 and 6 are diagrams for explaining an example of the extraction process of the pixel block. The pixel block extraction process can be performed, for example, as follows.
(1) Select n points (for example, 100 points) from the RGB image. FIG. 5 shows an example of n points selected from the RGB image 310. The + mark in FIG. 5 indicates a point to be selected. The dark-colored region 230i in FIG. 5 is a region showing the object 230. As shown in FIG. 5, the n points may be selected at regular intervals in each of the vertical direction and the horizontal direction, or may be randomly selected. In the example of FIG. 5, some of the n points are inside the region 230i indicating the object, and the remaining points are outside the region 230i. In this way, it is not necessary for all of the n selected points to be inside the region 230i indicating the object.
(2) A pixel block having a fixed area (for example, about 100 pixels) around each of the selected n points is extracted. FIG. 6 is a diagram showing an example of the extracted n pixel blocks. In FIG. 6, the area marked with □ indicates a pixel block. In the example shown in FIG. 6, none of the n pixel blocks overlaps with the other pixel blocks, but overlap may occur between the pixel blocks. Moreover, the size may be different for each pixel block.

See FIG. 4 again.

In step S103, the processor 112 smoothes (or averages) the image of each pixel block to determine representative values for the respective colors of R, G, and B. For example, the representative value of each color is determined by smoothing the image data of each of R, G, and B of each pixel block using a smoothing filter of a predetermined size. More specifically, the image data of each pixel block is smoothed by using a averaging filter having a size such as 3 × 3 pixels or 5 × 5 pixels, a weighted averaging filter, or a smoothing filter such as a Gaussian filter. it can. The smoothing process may be performed a plurality of times. By performing the smoothing, the values of R, G, and B of each pixel block are averaged. The processor 112 determines, for each pixel block, one averaged value for each of R, G, and B as a representative value. Instead of using the smoothing filter, the average value of each pixel value of R, G, and B of a plurality of pixels in each pixel block may be simply used as a representative value. In step S103, averaged data of R, G, and B is obtained for each of the plurality of pixel blocks.

ここで変換係数Ｋ_１１、Ｋ_１２、Ｋ_１３、Ｋ_２１、Ｋ_２２、Ｋ_２３、Ｋ_３１、Ｋ_３２、Ｋ_３３は、撮像システムに依存して適切な値に設定される。 In step S105, the processor 112, for each of the n pixel blocks, R, G, a representative value of each pixel value of ^{B, CIE-L * a *} b * L * in the color ^{system, a} *, b Convert to data of ^* . This conversion is performed, for example, as follows. The processor 112 first converts the data of R, G, and B into the data of X, Y, and Z in the CIE-XYZ color system. This transformation is performed using a known transformation matrix. For example, assuming that the values of red, green, and blue are R, G, and B, respectively, the values of X, Y, and Z can be obtained by the following conversion formulas 1 to 3.

Here, the conversion coefficients K ₁₁ , K ₁₂ , K ₁₃ , K ₂₁ , K ₂₂ , K ₂₃ , K ₃₁ , K ₃₂ , and K ₃₃ are set to appropriate values depending on the imaging system.

ここで、変換係数Ｋｌ、Ｋａ、Ｋｂ、Ｏｌは、撮像システムに依存して適切な値に設定される。Ｘ_０、Ｙ_０、Ｚ_０は、標準光の下での標準白色の三刺激値を表す。 Next, the processor 112 converts the data of X, Y, and Z into the data of L ^* , a ^* , and b ^* by the following conversion formulas 4 to 6.

Here, the conversion coefficients Kl, Ka, Kb, and Ol are set to appropriate values depending on the imaging system. X ₀ , Y ₀ , and Z ₀ represent standard white tristimulus values under standard light.

In this way, the processor 112 converts the values of R, G, and B in each pixel block into three values L ^* , a ^* , and b ^* in the L ^* a ^* b ^* color space, which are different from the RGB color space. .. Hereinafter, these three values will be referred to as L ^* a ^* b ^* data. By this conversion, the distance difference in the color space can be matched with the color difference perceived by humans regardless of the direction.

In step S105, the processor 112 sets the points on the L ^* a ^* b ^* color space indicated by the L ^* a ^* b ^* data and the L ^* a in all the other pixel blocks for each of the n pixel blocks. ^* B ^* Calculate the sum of the distances to the points in the L ^* a ^* b ^* color space indicated by the data. This process will be described with reference to FIGS. 7 and 8.

7 and 8 schematically show an example of a plurality of points on the L ^* a ^* b ^* color space indicated by the L ^* a ^* b ^* data of n pixel blocks. In this example, for the sake of simplicity, five points a to e are shown. In FIG. 7, the distance between the point a and all other points is indicated by an arrow. In FIG. 8, the distance between the point b and all other points is indicated by an arrow. Similarly, for the other points c, d, and e, the distance in the L ^* a ^* b ^* color space is defined. Processor 112 calculates the sum of the distances from all the other points for each of the n points in the L ^* a ^* b ^* color space. i-th among n pixel block (i is an integer from 1 to n) of the pixel block ^{L *,} a *, ^{b *} values, respectively _{L i,} a i, when the _{b i,} the processor 112 Calculates the total Si of the distances represented by the following equation 7 for all pixel blocks.

In step S106, the processor 112 selects m pixel blocks (m is an integer of 2 or more and less than n) in order from the n pixel blocks having the smallest total distance calculated in step S105. m can be set to a number of, for example, one to a few tenths of n. When n is, for example, 100, m can be set to a number of, for example, about 20.

FIG. 9 is a diagram showing an example of m points 320 having a relatively small total distance from n points on the L ^* a ^* b ^* color space to other points. As shown in FIG. 9, step S106 eliminates points having a relatively large sum of distances from other points, leaving only some points 320 having a relatively small sum of distances. By the above processing, the data of m points in the part where the points are dense on the L ^* a ^* b ^* color space is adopted. By setting the color space to L ^* a ^* b ^* , only approximate colors can be effectively extracted.

In step S107, the processor 112 calculates the average value of each of the values of L ^* , a ^* , and b ^{* in} the selected m pixel blocks, and records the average value in the memory 114 as a reference value. Instead of the average value, other representative values determined based on the values of L ^* , a ^* , and b ^* in the m pixel block may be recorded. Let the reference values of L ^* , a ^* , and b ^* recorded here be L _r , a _r , and b _r , respectively.

By the above processing, the registration of the color of the object 230 is completed. Recorded reference values _{L r,} a _{r, b} r represents a reference color of the object 230, is referenced in the determination process of the color of other objects 230 of the same type (step S20).

[2-2. Color judgment processing]
Subsequently, the color determination process of step S20 shown in FIG. 3 will be described.

FIG. 10 is a flowchart showing the flow of the color determination process. The color determination process includes steps S201 to S209. The processes of steps S201 to S207 are the same as the processes of steps S101 to S107 shown in FIG. 4, respectively. Therefore, detailed description of the processing of steps S201 to S207 will be omitted. The processor 112 calculates the average value of each of L ^* , a ^* , and b ^* from the RGB image data of the object 230 to be inspected by the processing of steps S201 to S207. Let the average values of the calculated L ^* , a ^* , and b ^{* be} the measured values L _m , a _m , and b _m , respectively. Thereafter, in step S208 and S209, the measured value _{L m,} a m, a _{b m,} prerecorded reference value _{L r,} a r, compares with the _{b r,} the quality of the color of the object 230 .. Hereinafter, the processes of steps S208 and S209 will be described in more detail.

In step S208, the processor 112, ^L * calculated at step ^S207, a *, ^{b *} of the respective average value _{L m,} and a point on a m, _{b m} shows the ^L * ^a * ^{b *} color space, prerecorded ^{L *,} a *, ^{b *} of each of the reference value _{L r,} calculates the distance between a point on a _{r, b} r shows the ^L * ^a * ^{b *} color space. This distance corresponds to the color difference and is represented by ΔE. The color difference ΔE is calculated by the following formula.

As described above, in the present specification, the "color difference" between two colors means the distance between the two colors in the color space. When the conversion from RGB to L ^* a ^* b ^* is performed as in the present embodiment, the distance ΔE in the L ^* a ^* b ^* color space corresponds to the color difference. If other transformations are performed, the distance in the other color space is calculated as the color difference. In addition, (ΔE) ² may be used as the color difference instead of ΔE.

In step S209, the processor 112 determines the quality of the color of the object 230 according to the calculated distance ΔE, and outputs the determination result. The determination result is displayed on the display 220. ΔE is a one-dimensional numerical value, and since the weights of each dimension are equal, it is excellent as a numerical value representing a color difference. Therefore, ΔE is convenient for measuring the difference from the reference color and determining pass / fail. Only the value of ΔE may be displayed on the display 220. However, by setting some threshold values based on this numerical value, pass / fail judgment or classification can be performed. For example, if ΔE is less than the threshold value, it can be regarded as “pass”, and if it is greater than or equal to the threshold value, it can be regarded as “fail”. Alternatively, if ΔE is less than the first threshold value, it is "rank A", if it is equal to or more than the first threshold value and less than the second threshold value, it is "rank B", and if it is equal to or more than the second threshold value and less than the third threshold value, it is "rank". Classification such as "Rank C" may be performed.

FIG. 11 is a diagram showing an example of a quality determination method. FIG. 11A shows an example of pass / fail determination based on a comparison between ΔE and one threshold value. FIG. 11B shows an example of rank determination based on a comparison between ΔE and a plurality of threshold values. In the example of (a), the threshold value is 1.0, and if ΔE <1.0, it is determined as “pass (OK)”, and if ΔE ≧ 1.0, it is determined as “fail (NG)”. In the example of (b), three thresholds 0.5, 1.0, and 1.5 are set, and if ΔE <0.5, “rank A” and 0.5 ≦ ΔE <1.0. For example, “Rank B” is determined, “Rank C” is determined if 1.0 ≦ ΔE <1.5, and “Failure (NG)” is determined if ΔE> 1.5. The number of thresholds may be four or more. The larger the number of thresholds, the more classes can be classified.

In the above example, each threshold value may be set manually, but it may not be easy to determine each threshold value because various conditions need to be considered. On the other hand, in the present embodiment, data of many points are obtained in the process of measuring and registering the reference color. Therefore, the color tolerance of the object to be measured can be known in advance from statistical data such as standard deviation. Therefore, each threshold can be determined based on statistical data such as standard deviation. As a result, a more appropriate quality judgment becomes possible.

FIG. 12 is a conceptual diagram showing an example of quality determination based on statistical data. In FIG. 12, two spheres 330 and 340 in the L ^* a ^* b ^* color space determined from the statistical data (that is, n L ^* a ^* b ^* data) obtained in step S104 or S204 are shown. Illustrated. The inner sphere 330 may be, for example, a sphere whose radius is the standard deviation σ of any of the statistical data of L ^* , a ^* , and b ^* . The outer sphere 340 may be, for example, a sphere having a radius 1.3 times the standard deviation σ of any one of the statistical data of L ^* , a ^* , and b ^* . Measurements L _m, a _m, if the point indicated by b _m is the interior of the inner sphere 330 "high rank products", "low ranks available if the inside of the outside and the outside of the sphere 340 of the inner sphere 330 If it is outside the outer sphere 340, it can be determined as "defective product". In the example of FIG. 12, two spheres 330 and 340 are defined in the L ^* a ^* b ^* color space, but even if one or more spheres or other solids (eg ellipsoids) are defined. Good. When one solid is defined, it can be determined that the point of the measurement data is "good" if it is inside the solid and "defective" if it is outside. Further, when three or more solids are defined, the object to be measured can be classified into four or more classes.

[2-3. How to determine the conversion factor]
Next, an example of a method for determining the conversion coefficients K11, K12, K13, K21, K22, K23, K31, K32, K33, Kl, Ka, Kb, and Ol in the above equations 1 to 6 will be described. In the present embodiment, these conversion coefficients are determined by using a machine learning algorithm such as an error back propagation method. The determination of the conversion coefficient can be performed in advance before the color registration process shown in FIG. A color chart in which a plurality of color samples are arranged in advance can be used to determine the conversion coefficient. For example, the Macbeth Color Checker shown in FIG. 13 can be used. In the Macbeth color checker, 24 rectangular color samples are arranged two-dimensionally as shown in the table. The color of each color swatch is known.

It suffices that such a color chart is imaged using a device that is actually used, and the color data obtained as a result matches the known color data. However, they usually do not match as they are and need to be adjusted. In the present embodiment, this adjustment is performed by changing each conversion coefficient in the above-mentioned equations 1 to 6 used at the time of color space conversion.

With ordinary methods, it is difficult to optimally adjust many coefficients for many colors. Therefore, in general, the adjustment is limited to a few typical colors (for example, red, green, blue, etc.). In this embodiment, at the time of this adjustment, machine learning is performed on a large number of known colors by using the backpropagation method. As a result, for many colors, it is possible to perform measurements that are close to the specified values or theoretical values.

An example of this adjustment method will be described below. In the following description, the main body of processing is the processor 112 in the signal processing device 110.
(A) First, an image is taken for one color sample in the color chart. Then, the same processing as in steps S101 to S103 shown in FIG. 4 is executed. As a result, the representative values of the R, G, and B colors are determined for each of the n pixel blocks.
(B) Next, each conversion coefficient in the above equations 1 to 6 is set to a temporary value, and the same processing as in steps S104 to S107 is executed. As a result, representative values of the respective values of L ^* , a ^* , and b ^* can be obtained. These representative values are called measured values.
(C) Next, the error between the measured values of L ^* , a ^* , and b ^* and the predetermined values of L ^* , a ^* , and b ^* of the color sample is calculated. The error is calculated, for example, by the same calculation as in Equation 8 described above.
(D) The above processes (a) to (c) are sequentially executed for all the other colors in the color chart. As a result, the error between the measured values of L ^* , a ^* , and b ^* and the specified values is calculated for each color.
(E) Calculate the sum or sum of squares of the errors obtained for each color.
(F) Each coefficient used in the color conversion process in (b) above is changed.

The processes (a) to (f) above are repeatedly executed. The adjustment of each coefficient in the above (f) is executed by using a machine learning algorithm such as an error back propagation method. Each coefficient is adjusted so that the sum of errors or the sum of squares obtained in (e) above becomes small. The coefficient used when the sum of errors or the sum of squares is the smallest is set as the coefficient actually used in steps S104 and S204.

By the above processing, the color identification accuracy can be significantly improved. FIG. 14 is a diagram showing the effect of the above coefficient adjusting method. FIG. 14 shows an example of the error between the measured value and the specified value of each color of the Macbeth color checker when machine learning is used and when it is not used. As shown in FIG. 14, by adjusting each coefficient using the above machine learning, the error of each color can be reduced to a fraction or less. As a result, the color discrimination accuracy can be significantly improved.

In the present embodiment, the processor 112 of the signal processing device 110 executes the adjustment of the coefficient using the above machine learning, but an external server may execute a part or all of this processing. Such a server may be connected to the signal processing device 110 via a network such as the Internet. By having an external server execute the determination of the coefficient, the scale of the signal processing device 110 can be reduced, and the imaging system 100 can be miniaturized.

In this embodiment, the signal processing unit 110 performs conversion into XYZ values from the RGB values, further performs conversion to ^L * ^a * ^{b *} values, and the measured ^L * ^a * ^{b *} value registered in advance The distance (that is, color difference) calculated from the calculated L ^* a ^* b ^* value is calculated. However, the conversion is not limited to the L ^* a ^* b ^* value, and the conversion to a value in another color space may be performed. For example, conversion to XYZ or L ^* u ^* v ^* may be performed, and the same processing as described above may be applied. In that case, the distance in the other color space may be treated as a color difference.

[3. effect]
As described above, according to the present embodiment, the signal processing device 110 performs the color registration process and the color determination process of the object 230. In the registration process, the signal processing device 110 determines n pixel blocks from the image indicated by the RGB image data acquired from the image pickup device 140, and represents the representative values of the respective pixel values of R, G, and B for each pixel block. (For example, the average value) is determined. Then, for each pixel block, the representative values of the respective pixel values of R, G, and B are set to three values (for example, L ^* a ^* b ^* color space) in a second color space different from the RGB color space. Convert to L ^* , a ^* , b ^* value). Further, for each pixel block, the point on the second color space indicated by the three values and the second color indicated by the three values for all the other pixel blocks contained in the n pixel blocks. Calculate the sum of the distances to points in space. A representative value (for example, an average value) of each of the above three values in the m pixel blocks selected in order from the one having the smallest total distance among the n pixel blocks is calculated and output as a reference value. The above process can be performed on all objects that require color registration. In the color determination process, the signal processing device 110 calculates the representative value of each of the three values in the second color space as the measured value in the same manner as described above for the object to be measured. Then, the distance between the point on the second color space indicated by the measured value and the point on the second color space indicated by the previously recorded reference value is calculated. The signal processing device determines whether the color of the object is good or bad according to the distance, and outputs the determination result.

According to the above processing, the representative values of each of the above three values in the m pixel blocks selected in order from the one having the smallest total distance among the n pixel blocks are calculated. As a result, only some pixel blocks having similar colors are selected from the n pixel blocks, and representative values of the above three values are determined based only on the data of the part of the pixel blocks. By such a process, it is possible to more accurately identify or evaluate the color of an object whose surface color is not uniform, such as a cloth or paper having a tint block or a crease (particularly one having a large fiber).

In the conventional color measurement, the measurement result is often expressed by a standard color system having three or more dimensions such as an XYZ color system or a Lab color system. However, such expressions are generally difficult to understand and it is difficult to evaluate the color of the object. For example, when the color of a product is measured in an inspection process at a manufacturing site and the pass / fail judgment of the product is made based on the measurement result, it is difficult to judge the pass / fail by the conventional method, and it is also difficult to give feedback to the manufacturing process. According to the present embodiment, the judgment result of the quality of the color of the object is displayed in an easy-to-understand manner such as "pass", "fail", or "rank A", "rank B", "rank C". To. Therefore, the color of the product can be easily evaluated, and for example, feedback to the manufacturing process becomes easy.

Further, according to the present embodiment, the conversion coefficient used when converting the representative values of the respective pixel values of R, G, and B into the three values in the second color space is determined by machine learning. .. As a result, the color discrimination performance can be significantly improved.

Conventionally, in color measurement where accuracy is required, it is required to use a camera or illumination having a predetermined color filter. For example, in the shooting of the three primary colors R, G, and B, the wavelength of each color is specified. ], Green (G): 546.1 [nm], Blue (B): 435.8 [nm]. Originally, the conversion from RGB to XYZ and the conversion from XYZ to L ^* a ^* b ^* are performed based on these preconditions.

Since the purpose of this technology is not to evaluate the absolute value of color, it is not necessary to accurately meet these standards. However, in order to measure a wide range of colored objects with uniform sensitivity, it can be said that measurement using these standard wavelengths is desirable.

On the other hand, generally used physical devices such as LEDs and cameras do not always have the characteristics as specified values. Those close to the specified value are generally large and tend to be expensive, and it is difficult to approach the ideal theoretical value, especially in a limited cost and size. It is possible to take a picture of a known color with an actual device and correct each coefficient of the conversion formula based on the result, but there are many parameters and it is difficult to match with various colors. For this reason, it is often performed only for typical colors and colors that are frequently used.

On the other hand, in the method of the present embodiment, each coefficient of each conversion formula is optimized by using a machine learning algorithm such as an error back propagation method. Therefore, the optimum coefficient can be determined without depending on the device actually used.

[4. Modification example]
Next, a modified example of the above embodiment will be described.

In the above embodiment, as an image, an RGB image including a plurality of pixels, each of which has three pixel values corresponding to three colors of R, G, and B, is used, which is generated by imaging an object. Be done. However, the image is not limited to an RGB image, and may be a single color, two colors, or an image of four or more colors. In the above embodiment, the representative value of the pixel value of each color in each pixel block is a second color space (for example, L ^* a ^* b ^* color) different from the first color space composed of R, G, and B. It is converted to space etc.). However, such conversion processing may be omitted. Even in that case, for each pixel block, the representative value of each pixel value of one or more colors and the one or more colors for all the other pixel blocks included in the n pixel blocks. The sum of the values indicating the degree of difference between each pixel value and the representative value is calculated. Then, among the n pixel blocks, each of one or more colors in the m pixel blocks (m is an integer of 2 or more and less than n) selected in order from the one having the smallest sum of the values indicating the degree of the difference. A value indicating one color is determined based on the representative value of the pixel value of. By such a process, the color or density of the object can be appropriately evaluated.

The technique of the present disclosure can be used to identify the color of an object. For example, it can be used for inspecting the color of a product having a non-uniform surface color.

100 Imaging system 110 Signal processing device 112 Processor 114 Memory 116 Input / output interface 120 Lighting device 140 Imaging device 160 Control device 220 Display 230 Object

Claims (16)

A step of acquiring image data containing a plurality of pixels, each of which has one or more pixel values corresponding to one or more colors.
A step of extracting n pixel blocks (n is an integer of 3 or more) from the image, and
For each of the extracted n pixel blocks, a step of determining a representative value of each pixel value of the one or more colors, and
For each of the n pixel blocks, a representative value of each pixel value of the one or more colors and the one or more colors for all the other pixel blocks included in the n pixel blocks. The step of calculating the sum of the values indicating the degree of difference between the representative values of each pixel value and
Each of the one or more colors in the m pixel blocks (m is an integer of 2 or more and less than n) selected in order from the one having the smallest sum of the values indicating the degree of the difference among the n pixel blocks. The step of determining and outputting a value indicating one color based on the representative value of the pixel value of
How to quantify the color of an object, including. The method according to claim 1, wherein the one or more colors are a plurality of colors. The method of claim 2, wherein the plurality of colors include red (R), green (G), and blue (B). The step of calculating the sum of the values indicating the degree of the difference is
For each of the n pixel blocks, the representative value of each pixel value of the plurality of colors is set to a plurality of values in a second color space different from the first color space composed of the plurality of colors. Steps to convert and
For each of the n pixel blocks, the points on the second color space indicated by the plurality of values and the plurality of values for all the other pixel blocks included in the n pixel blocks are The step of calculating the sum of the distances to the points on the second color space shown as the sum of the values indicating the degree of the difference, and
Including
The value indicating one color is a representative value of each of the plurality of values in the m pixel blocks selected in order from the n pixel blocks having the smallest total distance.
The method according to claim 2 or 3. The one or more colors include red (R), green (G), and blue (B).
The image data is RGB image data that includes a plurality of pixels, each of which has red (R), green (G), and blue (B) pixel values, generated by imaging an object. ,
Each pixel value of the one or more colors is a pixel value of R, G, and B, respectively.
The step of calculating the sum of the values indicating the degree of the difference is
For each of the n pixel blocks, the representative values of the respective pixel values of R, G, and B are set in a second color space different from the first color space composed of R, G, and B. Steps to convert to 3 values and
For each of the n pixel blocks, the points on the second color space indicated by the three values and the three values for all the other pixel blocks included in the n pixel blocks are The step of calculating the sum of the distances to the points on the second color space shown as the sum of the values indicating the degree of the difference, and
Including
The value indicating one color is a representative value of each of the three values in the m pixel blocks selected in order from the n pixel blocks having the smallest total distance.
The method according to claim 1. The point on the second color space indicated by the representative value of each of the three values in the m pixel block, and the second color indicated by the reference value of each of the three values recorded in advance. Steps to calculate the distance to a point in space,
The method according to claim 5, further comprising a step of determining the quality of the color of the object according to the distance and outputting the determination result. The step of calculating and outputting the representative value of each of the three values further includes recording the representative value of each of the three values on a recording medium as a reference value.
The method further
The step of acquiring the data of another RGB image generated by imaging another object, and
A step of extracting another n pixel blocks from the other RGB image, and
For each of the extracted other n pixel blocks, a step of determining a representative value of each pixel value of R, G, and B, and
A step of converting the representative values of the respective pixel values of R, G, and B into three values in the second color space for each of the other n pixel blocks.
For each of the other n pixel blocks, the point on the second color space indicated by the three values and the said for all other pixel blocks contained in the other n pixel blocks. A step of calculating the sum of the distances to the points on the second color space indicated by the three values, and
A step of calculating a representative value of each of the three values in the m pixel blocks selected in order from the one having the smallest sum of the distances among the other n pixel blocks.
The point on the second color space indicated by the representative value of each of the three values in the other m pixel blocks, and the reference value of each of the three values recorded in advance indicates the first. Steps to calculate the distance to a point in the color space of 2 and
A step of determining the quality of the color of the other object according to the distance and outputting the determination result,
5. The method of claim 5, further comprising. The method according to any one of claims 5 to 7, wherein the three values in the second color space are L ^* value, a ^* value, and b ^* value in the L ^* a ^* b ^* color space. A claim that further includes a step of determining by machine learning the conversion coefficient used when converting the representative values of the respective pixel values of R, G, and B into the three values in the second color space. Item 8. The method according to any one of Items 5 to 8. Any of claims 5 to 9, wherein the step of determining the representative value of each of the R, G, and B pixel values includes a step of smoothing and averaging the respective pixel values of R, G, and B. The method described in. With the processor
A memory that stores computer programs executed by the processor, and
With
The computer program is attached to the processor.
A step, each of which obtains data of images including a plurality of pixels having one or more pixel values corresponding to one or more colors,
A step of extracting n pixel blocks each containing n pixels (n is an integer of 3 or more) separated from each other from the image, and a step of extracting the n pixel blocks.
For each of the extracted n pixel blocks, a step of determining a representative value of each pixel value of the one or more colors, and
For each of the n pixel blocks, a representative value of each pixel value of the one or more colors and the one or more colors for all the other pixel blocks included in the n pixel blocks. The step of calculating the sum of the values indicating the degree of difference between the representative values of each pixel value and
Each of the one or more colors in the m pixel blocks (m is an integer of 2 or more and less than n) selected in order from the one having the smallest sum of the values indicating the degree of the difference among the n pixel blocks. A step of calculating and outputting a value indicating one color based on the representative value of the pixel value of
A signal processing device that executes. The signal processing device according to claim 11, wherein the one or more colors are a plurality of colors. The signal processing apparatus according to claim 12, wherein the plurality of colors include red (R), green (G), and blue (B). The step of calculating the sum of the values indicating the degree of the difference is
For each of the n pixel blocks, the representative value of each pixel value of the plurality of colors is set to a plurality of values in a second color space different from the first color space composed of the plurality of colors. Steps to convert and
For each of the n pixel blocks, the points on the second color space indicated by the plurality of values and the plurality of values for all the other pixel blocks included in the n pixel blocks are The step of calculating the sum of the distances to the points on the second color space shown as the sum of the values indicating the degree of the difference, and
Including
The value indicating one color is a representative value of each of the plurality of values in the m pixel blocks selected in order from the n pixel blocks having the smallest total distance.
The signal processing device according to claim 12 or 13. The one or more colors include red (R), green (G), and blue (B).
The image data is RGB image data that includes a plurality of pixels, each of which has red (R), green (G), and blue (B) pixel values, generated by imaging an object. ,
Each pixel value of the one or more colors is a pixel value of R, G, and B, respectively.
The step of calculating the sum of the values indicating the degree of the difference is
For each of the n pixel blocks, the representative values of the respective pixel values of R, G, and B are set in a second color space different from the first color space composed of R, G, and B. Steps to convert to 3 values and
For each of the n pixel blocks, the points on the second color space indicated by the three values and the three values for all the other pixel blocks included in the n pixel blocks are The step of calculating the sum of the distances to the points on the second color space shown as the sum of the values indicating the degree of the difference, and
Including
The value indicating one color is a representative value of each of the three values in the m pixel blocks selected in order from the n pixel blocks having the smallest total distance.
The signal processing device according to claim 11. The signal processing device according to any one of claims 11 to 15.
An imaging apparatus for generating data of the image by imaging the Target material,
Imaging system with.

Images