EP3759445A1

EP3759445A1 - Method for setting colorimetric conversion parameters in a measuring device

Info

Publication number: EP3759445A1
Application number: EP19710505.9A
Authority: EP
Inventors: Takuroh Sone; Hideyuki Kihara; Takashi Soma; Shuhei WATANABE; Takafumi HIROI
Original assignee: Ricoh Co Ltd
Current assignee: Ricoh Co Ltd
Priority date: 2018-03-02
Filing date: 2019-02-21
Publication date: 2021-01-06
Also published as: CN111801554A; JP2019153931A; WO2019167806A1

Abstract

Disclosed is a measuring device for precisely converting photographing data into tristimulus values so as to correspond to sensitivity characteristics of human vision. The measuring device includes at least one lighting unit configured to irradiate the object with light, at least one photographing unit configured to photograph the object irradiated with the light to produce a captured image, and a converter configured to convert the captured image into tristimulus values.

Description

[Title established by the ISA under Rule 37.2] METHOD FOR SETTING COLORIMETRIC CONVERSION PARAMETERS IN A MEASURING DEVICE

The disclosure discussed herein relates to a measuring device that measures appearance characteristic, a method for setting colorimetric conversion parameters in the measuring device, and an industrial product inspected by the measuring device.

An appearance of industrial products, e.g., a texture is an important factor to affect purchase incentive. To manage and improve a quality of the texture, a texture evaluation may be important. However, evaluating the texture by visual inspection creates the problem of variation in evaluation. For this reason, a measuring instrument is needed to digitalize the texture.

In viewing an appearance of the product, people observe the product from different angles since its color or luster depends on a viewing angle. When digitalizing the texture, it is thus necessary to measure the product at a plurality of angle conditions. In view of such a need, a colorimeter, which can change a lighting angle or a light receiving angle for measurement of the production, is commercially available. However, the colorimeter can only measure an average color within a measurement range around a minute point, and is unable to measure texture information with respect to a target plane. The "texture information" relates to patterns, granularity, and glitter of a sample surface. Without such texture information, it is difficult to digitalize the appearance. Accordingly, it is preferable that the sample surface is imaged by a camera.

When such image data is obtained by a combination of multiple light sources and a color camera, the obtained RGB image data needs to be converted into tristimulus values XYZ corresponding to the sensitivity of human vision, or an L*a*b* value in an L*a*b* color system. However, even if the RGB data is converted into tristimulus values XYZ or an L*a*b* value by using a general conversion formula to digitalize the texture, such RGB does not correspond to the sensitivity of human vision. As a result, the digitalized data is deviated from human perception.

Accordingly, for purposes of converting the RGB data into the tristimulus values XYZ, as a method of converting the RGB data into the tristimulus values XYZ corresponding to a color sensitivity of a person, Patent Document 1 discloses a convert method with use of a previously obtained multi-grid 3D-LUT (Look-Up Table for color conversion).

[PTL 1] Japanese Unexamined Patent Application Publication No. 2009-239419

The method of the Patent Document 1 may be used to measure a sample with large angle dependency, such as a sample having a coating with luster finish that may make its color vary in response to the lighting angle or the light receiving angle. In such a case, an error in conversion may be increased if the 3D-LUT specific for a certain angle is applied to other angles in converting the RGB data into the tristimulus values XYZ.

In view of the above-described problems, one aspect of the present invention is directed to providing a measuring device whereby it is possible to conduct photographing multiple times under multiple setting conditions to precisely convert photographing data into tristimulus values, which include color information corresponding to sensitivity characteristics of human vision, in each setting condition.

According to one embodiment of the present invention, a measuring device for measuring an object to be measured is provided so as to conduct photographing multiple times under multiple setting conditions to precisely convert photographing data into tristimulus values corresponding to sensitivity characteristics of human vision. The measuring device includes:
at least one lighting unit configured to irradiate the object with light;
at least one photographing unit configured to photograph the object irradiated with the light to produce a captured image; and
a converter configured to convert the captured image into tristimulus values,
wherein the photographing unit is configured to conduct photographing multiple times by use of multiple setting conditions for changing at least one of a lighting angle of the lighting unit and a photographing angle of the photographing unit, and
　　　　　　in the converter, a condition for converting the captured image into the tristimulus values is different for each of the setting conditions.

According to an aspect of the embodiments, it is possible to provide a measuring device for photographing multiple times under multiple setting conditions so as to precisely convert photographing data into tristimulus values, which include color information corresponding to sensitivity characteristics of human vision, in each setting condition.

FIG. 1 is a diagram illustrating an example of an appearance characteristics measuring system according to a first embodiment; FIG. 2 is a block diagram illustrating an example of a hardware configuration of the appearance characteristic measuring system; FIG. 3 is a functional block diagram illustrating an example of an information processing apparatus of Fig. 1; FIG. 4 is an entire flowchart illustrating an example of a colorimetry process according to the first embodiment; FIG. 5A is a diagram for explaining differences of reflection by material of an object to be measured; FIG. 5B is a diagram for explaining differences of reflection by material of an object to be measured; FIG. 5C is a diagram for explaining differences of reflection by material of an object to be measured; FIG. 5D is a diagram for explaining differences of reflection by material of an object to be measured; FIG. 6 is a diagram for explaining a state when colorimetry conversion parameters are initialized upon shipping; FIG. 7 is a flowchart illustrating an example of initially setting the colorimetry conversion parameters; FIG. 8 is a flowchart illustrating an example of updating the colorimetry conversion parameters; FIG. 9 is a diagram illustrating values converted to tristimulus values XYZ from RGB data with respect to shade for 12-color tiles to be used, according to a parameter color conversion formula created based on a shading condition; FIG. 10 is a diagram illustrating values converted to tristimulus values XYZ from RGB data with respect to highlight for 12-color tiles to be used, according to a parameter color conversion formula created based on a highlight condition; FIG. 11 is a detailed flowchart illustrating an example of creating composited RGB data in a colorimetry process according to a second control example; FIG. 12 is a diagram illustrating a configuration example of an integrated appearance characteristic measuring apparatus; FIG. 13 is a diagram illustrating an example of an appearance characteristic measuring system according to the second embodiment; FIG. 14 is a diagram illustrating an example of an appearance characteristics measuring system according to a modification of the second embodiment; and FIG. 15 is a diagram illustrating a spectral line of a spectral camera included in an appearance characteristic measuring system according to a third embodiment.

The following illustrates embodiments for carrying out the present invention with reference to the accompanying drawings. Identical reference numerals are used to denote identical components in each drawing; accordingly, for the identical components, explanation may be omitted.

FIRST EMBODIMENT
Referring to Figs 1 to 3, explanation will be provided for a configuration of an appearance characteristic measuring apparatus according to a first embodiment.

The appearance characteristic measuring apparatus of the present embodiment irradiates a surface of an object with light at multiple angles to photograph the object surface with a photographing unit. The appearance characteristic measuring apparatus also converts the photographed image data into XYZ data (measured values) and values indicating the texture by using a previously obtained color conversion formula in each lighting angle or light receiving angle, and outputs the XYZ data and the texture values. The following illustrates a configuration for achieving such functionality.

FIG. 1 is an overall schematic diagram illustrating an example of an appearance characteristic measuring system 100 according to the first embodiment. FIG. 2 is a block diagram illustrating a hardware configuration of the appearance characteristic measuring system 100.

As illustrated in Figs. 1 and 2, the appearance characteristic measuring system 100 includes a light source 1, a photographing device 2, an inspection table 3, an information processing apparatus 4, and a monitor 5. The appearance characteristic measuring system 100 is a measuring device according to the present embodiment.

In the present embodiment, the light source 1 includes two lighting units 11 and 12 so as to irradiate a sample S as a measuring object with light shone at two or more lighting angles. The sample S is disposed on the inspection table 3.

In the present embodiment, a surface-mount-type white LED (Light Emitting Diode) with high color-rendering properties is used as each of the lighting units 11 and 12. A color rendering index of the LED exceeds 95. In general, an LED has low color rendering properties due to a specific spectral shape of the LED. This results in different colors when viewed under the sun (i.e., under natural light), thereby resulting in a failure to represent true colors. In contrast, the LED of the present embodiment has high color-rendering properties so as to improve color conversion accuracy.

A first lighting unit 11 is disposed at an angle of 15 degrees from a regular reflection direction with respect to the photographing unit 21. A second lighting unit 12 is disposed at an angle of 45 degrees from the regular reflection direction.

Such configuration allows the first lighting unit 11 to irradiate the sample S with the light so as to be reflected on the sample S in a vicinity of the regular reflection direction with respect to the photographing unit 21 (a highlight condition). Further, such configuration allows the second lighting unit 12 to irradiate the sample S with the light so as to be reflected on the sample S in a diffusion direction (a shade condition).

Note that this example illustrates the highlight condition and the shading condition. However, the arrangement of the lighting units 11 and 12 and the photographing unit 21 may be changed. For example, one or more lighting units and one or more photographing units may be changeably disposed so as to conduct photographing as long as the following two types of conditions are met. A first condition is a condition for irradiating the sample S with light from a first angle with respect to the photographing unit 21. A second condition is a condition for irradiating the sample S with light from a second angle with respect to the photographing unit 21. The second angle is different from the first angle.

The photographing device (imaging device) 2 includes a photographing unit (camera) 21. The photographing device 2 conducts photographing to obtain image data (RGB: Raw data) for the sample S disposed on the inspection table 3.

In the present embodiment, the lighting units 11 and 12, and the camera 21 are supported by a circular baseplate 8.

In the present embodiment, a camera with a Bayer RGB array is used as the camera of the photographing unit 21. In the Bayer array, photodiodes of the camera are arranged such that columns of arranged R (red) filters and G (green) filters, and columns of arranged G (green) filters and B (blue) filters are disposed alternately.

The photographing unit 21 is capable of photographing a portion of a surface of the sample S at a time. In this case, for example, the size of the portion is several tens of mm by several tens of mm (e.g., 50mm by 50mm).

Further, the camera of the photographing unit 21 can obtain R, G, and B each in 10 bits. In this case, for example, this photographing unit 21 adjusts a focus and a working distance of the camera such that resolution of the photographed image data can be 20μm per pixel.

The information processing apparatus 4 has a colorimetric value conversion function and a texture operation function, for calculating colorimetric values based on the image data (RGB: Raw data) for each of the setting conditions.

Note that in FIG. 1 the information processing apparatus 4 is apart from the lighting units 11 and 12 and the photographing unit 21. However, a function of the color operation unit implemented by the information processing apparatus 4 may be implemented by an apparatus that has a housing for covering the lighting units 11 and 12 and the photographing unit 21 for integrating. Such a configuration will be described below in conjunction with FIG. 12.

Alternatively, a function of the colorimetric processing unit (information processing apparatus) serving as a color operation unit may be implemented by a computing device (information processing apparatus) such as a separate computer, which is totally independent of the lighting units 11 and 12 or the photographing unit 21.

The monitor 5 displays a photographed image and information on tristimulus values and texture.

In the present embodiment, a plurality of light sources (lighting units 11 and 12) are provided to allow the sample S to be irradiated with light from at least two lighting angles. In this example, light can be emitted from two lighting angles. The emitting is not performed at once from two directions, but is performed from a single lighting angle per shot.

Note that the sample S of this embodiment is disposed on the inspection table 3 as an example. However, the inspection table 3 may be a conveyor belt, for example. In this case, for example, the sample S, which is an industrial product conveyed in e.g., a direction perpendicular to the drawing sheet for Fig. 1, is temporarily stopped. The sample S is then photographed by the appearance property measuring system 100 of Fig. 1 with light from a plurality of irradiation directions, or from a plurality of photographing directions. This allows a color (measuring value) and texture of the industrial product to be inspected during manufacture.

The industrial product is a processed product made of metal material, non-metal material, material that is a combination of metal material and non-metal material, or the like. The industrial product means a product that is subjected to a surface processing. Examples of the industrial product include motor vehicles including a two-wheeled vehicle and a four-wheeled vehicle, a rolling stock such as a railway vehicle, a sheet metal that is used in the rolling stock, interior parts such as a car seat or a car dash board. Further, the examples of the industrial product include an aircraft, a vessel, a construction material, a building including the construction material, a photographing device, an information processing apparatus such as a personal computer, a mobile terminal such as a smartphone or a tablet, a home appliance such as a watch, a television apparatus, a refrigerator or an air conditioner, a cooking equipment such as a dish or a pot, and the like. Any industrial product can be subjected to measurement as long as the characteristics of the outer surface can be measured.

Referring to FIG.2, the lighting device 1 includes a first lighting unit 11, a second lighting unit 12, and a lighting controller 13 that drives each of the lighting units 11 and 12 for emitting light. The first lighting unit 11 and the second lighting unit 12 correspond to a plurality of lighting units. Note that FIG. 2 illustrates a case that the lighting controller 13 is shared by the lighting units 11 and 12, but may be separately provided to each of the lighting units 11 and 12.

The photographing device 2 includes one photographing unit (camera) 21 and an imaging processor 22. The photographing device 2 obtains images, each at a single photographing operation (one shot), by use of the two respective irradiation angles (lighting angles) of the lighting units 11 and 12 of the light source 1. The two irradiation angles are set to different angles.

A general computer device may be used as the information processing apparatus 4. Specifically, the computer device may be dedicated in the appearance characteristic measuring apparatus 100 of the present embodiment. Alternatively, an external computer may be used for colorimetric value conversion by loading a colorimetric value conversion program.

Referring to FIG.2, the information processing apparatus 4 includes a CPU (Central Processing Unit) 41, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 43, and a HDD (Hard Disk Drive) 44. The information processing apparatus 4 also includes various interfaces (I/F) 45, an Input and Output controller and an Input and Output (I/O) interface 46. The CPU 41, the ROM 42, the RAM 43, the HDD 44, the I/F 45, and the I/O interface 46 are connected to each other via a bus line 47.

In order to measure a surface of the measuring object such as a sample or a patch, the HDD 44 stores programs for a photographing control of the photographing device 2 and a lighting control of the light source 1. The HDD 44 also stores a texture calculation program as well as a colorimetric value conversion program for performing colorimetric value conversion or the like by use of the obtained RGB and Raw data.

A liquid crystal display can be used as the monitor 5, for example. The monitor 5 is capable of displaying image data or a calculation result as well as a setting menu, an operation menu, and the like. The monitor 5 is also capable of displaying RGB images, RGB values, calculated values of the tristimulus values XYZ, L*a*b* values, and L*a*b* dispersion values, which correspond to each of a highlight condition and shading condition used for imaging, in addition to various reference graphs or images or the like generated based on these values.

For example, simulations or the like are used as the reference graphs or images. The simulations indicate color visions modeled by use of each of the lighting units 11 and 12 of the light source 1, based on chromaticity diagrams corresponding to the tristimulus values XYZ, and coordinate positions in an L*a*b* color space, and L*a*b* color values.

INFORMATION PROCESSING APPARATUS
FIG. 3 is a functional block diagram illustrating an example of an information processing apparatus 4 of the appearance characteristic measuring apparatus 100. Note that, in FIG. 3, functional blocks of FIG. 3 are denoted by a solid line, and blocks related to control example to be described later are denoted by a broken line.

In FIG. 3, multiple functional blocks are implemented by the CPU 41 executed according to the colorimetric value conversion program. The functions of FIG.3 relate to a colorimetry process of the information processing apparatus 4.

The colorimetric value conversion program may be recorded by a computer-readable medium recorded in an installable format or an executable file format, such as a CD-ROM (Compact Disk Read Only Memory) or a flexible disk (FD). A CD-R (Compact Disk Recordable), a DVD (Digital Versatile Disk), a blue ray disk, a semiconductor memory or the like may be used as a computer readable recording medium. The colorimetric value conversion program may be installed via a network such as the Internet, or may be incorporated into a ROM or the like provided with an apparatus.

The information processing apparatus 4 includes a data input unit 80A, a lighting controller 81, a photographing controller 82, an image data storage 83, a calculation data storage 84, and a colorimetric conversion parameter updating unit 85. Further, the information processing apparatus 4 includes a colorimetric value calculator 86, a texture calculator 87, a measuring data storage 88, a communication unit 89, and a monitor output 80B.

The CPU 41 of FIG. 2 implements the functions of the lighting controller 81, the photographing controller 82, the colorimetric conversion parameter updating unit 85, the colorimetric value calculator 86, and the texture calculator 87, as illustrated in FIG.3. In the following description, these functions are implemented by software processing. However, all or part of the lighting controller 81, the photographing controller 82, the colorimetric conversion parameter updating unit 85, the colorimetric value calculator 86 and the texture calculator 87 may be implemented by hardware processing.

The image data storage 83, the calculation data storage 84, and the measuring data storage 88 are implemented by any of the HDD 44, the ROM 42 and the RAM 43 of FIG. 2, and an EEPROM (Electrically Programmable Read-only Memory).

The data input 80A, the communication unit 89, and the monitor output 80B are implemented by any of the various interfaces (I/F) 45, the Input and Output controller, and the Input and Output (I/O) interface 46, and the like.

Referring to FIG.3, the lighting controller 81 selectively controls the lighting or lighting off of the lighting units 11 and 12.

The photographing controller 82 causes the photographing unit 21 to perform photographing at a predetermined timing after the lighting unit 11 or 12 is lighted.

The image data storage 83 includes at least a highlight image storage 831 and a shade image storage 832, for storing photographed images for colorimetric value calculation.

The calculation data storage 84 stores data, which is referenced by the colorimetric value calculator 86 and the texture calculator 87.

Specifically, the calculation data storage 84 includes a noise processing data storage 841, an RGB combination data storage 842, a conversion formula for highlight calculation storage (highlight conversion formula storage) 843, a conversion formula for shade calculation storage (shade conversion formula storage) 844, an L*a*b* calculation data storage 845, an original data of conversion formula storage (original data storage) 846, and a fixed true value storage 847.

The colorimetric conversion parameter updating unit 85 updates parameters, which are stored in the highlight conversion formula storage 843 and the shade conversion formula storage 844 and are substituted into the conversion formulas (or conversion tables). The process of updating the parameters will be described later in conjunction with FIGs. 6 through 8.

The colorimetric value calculator (convertor) 86 includes at least a calibration processor 861, a demosaicing processor 862, and a tristimulus values XYZ calculator 864.

The calibration processor 861 performs calibration for correcting distortion of image, which may result from a lens of the photographing unit 21.

The demosaicing processor 862 performs demosaicing for changing Bayer-array of a raw image into a general RGB-array.

For a highlight image in the highlight image storage 831, the tristimulus values XYZ calculator 864 converts RGB data (image) per pixel into tristimulus values XYZ by use of a highlight conversion formula (color conversion formula or colorimetric value conversion formula) stored in the highlight conversion formula storage 843.

For a shade image in the shade image storage 832, the tristimulus values XYZ calculator 864 converts RGB data (image) per pixel into tristimulus values XYZ by use of a shading conversion formula (color conversion formula or colorimetric value conversion formula) stored in the shade formula storage 844.

Note that the colorimetric value calculator 86 may include a combination RGB data creating unit (RGB data combining unit) 863. The process of creating the combined RGB data will be described later in conjunction to FIG. 11.

The texture calculator 87 includes an L*a*b* calculator 871 and an L*a*b* dispersion value calculator 872.

The L*a*b* calculator 871 calculates L*a*b* color values per pixel of images obtained under each of the highlight condition and the shade condition, based on the corresponding tristimulus values XYZ calculated by the tristimulus values XYZ calculator 864. The L*a*b* color values are numerical values of color.

The L*a*b* dispersion value calculator 872 calculates dispersion values of the L*a*b* color values (e.g., variances) to obtain texture data.

The measuring data storage 88 stores the tristimulus values XYZ per pixel calculated by the colorimetric calculator 86, the L*a*b* color values per pixel calculated by the texture calculator 87, and the dispersion values of the L*a*b* color values.

The communication unit 89 transmits measuring data to another device (for example, another information processing apparatus), which is coupled to the information processing apparatus 4 by wire or wireless. The communication unit 89 also communicates with a host system 9 of FIG. 6.

The monitor output 80B outputs photographed images and measurement data, such as the tristimulus values XYZ, the L*a*b* chromatic values, and the L*a*b* dispersion values, with a display format of the monitor 5.

FIRST PROCESS OF CALCULATING COLORIMETRIC VALUES
As an example, FIG. 4 is an overall flowchart illustrating of a process of calculating colorimetry values according to a first example.

In S1, the sample S is set on the inspection table 3.

In S2, the first lighting unit 11 is lighted.

In S3, the photographing unit 21 photographs the sample S. This allows an RGB image (Raw data) with respect to the sample S to be obtained with use of a lighting angle of the highlight condition by the photographing unit 21. The photographed data (captured data) is, as Raw data having luminance information, stored in the highlight image storage 831.

In S4, the first lighting unit 11 is lighted off, while the second lighting unit 12 is lighted.

In S5, the photographing unit 21 photographs the sample S. This allows an RGB image (Raw data) with respect to the sample S to be obtained with use of a lighting angle of the shade condition by the photographing unit 21. The photographed data is, as Raw data having luminance information, stored in the shade image storage 832.

In S6, after performing noise reduction processing and smoothing processing of the Raw data obtained in S3 and S5, the calibration processor 861 performs calibration of the images to be processed by use of measuring data with respect to a white reference plate.

In S7, the demosaicing processor 862 demosaics the obtained data by use of, for example, advanced-color-plane interpolation or the like. Note that the order of S6 and S7 may be reversed.

The above processes can obtain two types of RGB data, i.e., highlight RGB luminance data (which corresponds to RGB data obtained in S3) and shade RGB luminance data (which corresponds to RGB data obtained in S5).

In S8, the tristimulus values XYZ calculator 864 converts the highlight RGB luminance data into the tristimulus values XYZ by use of the color conversion formula set under the highlight condition to create highlight XYZ data. Also, the tristimulus values XYZ calculator 864 converts the shade RGB luminance data into tristimulus values XYZ by use of the color conversion formula set under the shade condition to create shade XYZ data. The processes of S6, S7 and S8 are referred to as a process of calculating colorimetric values, where the S6 process is a main process of this calculating process.

In S9 and S10, the texture of the sample S is digitalized based on the above XYZ data (process of digitalizing texture).

Specifically, in S9, each of the highlight XYZ data and the shade XYZ data is converted into the L*a*b* data by use of a conversion formula formulated by International Commission on Illumination (CIE).

Note that the texture may be directly digitalized by use of the above XYZ data, but the XYZ data is deviated from human perception. For this reason, in the present embodiment, in S9, the texture is digitalized after the XYZ data (XYZ image) is converted into values in the L*a*b* color system. The conversion formula used in S9 is expressed as Formula 1 below.

In Formula 1, X₀, Y₀, and Z₀ denote the tristimulus values X, Y, and Z obtained by use of the white reference plate.

Now, "glitter" specific for a surface of a portion coated with bright material is a kind of texture. An example of digitalizing the glitter will be described below. A high glitter means that color of an image varies.

In view of the above point, in S10, the L*a*b* dispersion data is calculated for each of the highlight L*a*b* data and the shade L*a*b* data. The L*a*b* dispersion data for all pixels of the image to be processed allows the glitter to be digitalized. In the present embodiment, the glitter is indicated by use of the result of performing one or more of the four basic arithmetic operations with respect to the L*a*b* dispersion values, e.g., a product of the L*a*b* dispersion values (i.e., a value obtained by multiplying the dispersion values).

For example, when a measuring object is of a solid color, chromaticity of the object does not change. The product in this case is thus relatively large. On the other hand, when the surface of the measuring object has glitter, chromaticity of the object image changes largely. The product in this case is relatively small. In this matter, the glittering can be evaluated by use of a change amount in luminance.

In S11, the L*a*b* dispersion values are outputted as texture.

According to this flowchart, in S3 and S5, the measuring object is photographed from the different lighting angles. In S8, the RBG luminance data obtained under setting-condition-specific color conversion is converted into the XYZ data (colorimetric values). Further, in S10, the XYZ data is converted into the value indicating texture. Accordingly, the photographed data can be accurately converted into the tristimulus values XYZ, i.e., color information corresponding to the sensitivity of human color vision.

In S9, the tristimulus values XYZ are converted into the chromatic values in the L*a*b* color system (see Formula 1), whereby it is possible to digitalize the texture accurately.

In S10, the dispersion value for L*a*b* is calculated, whereby it is possible to digitalize, as the texture, "glitter" related to properties of particles, such as particle colors.

TEXTURE OF MEASURING OBJECTION
With reference to FIG. 5, explanation will be provided below for the glittering with respect to the measuring object. FIGs. 5A through 5D are diagrams for explaining differences of reflection of the measuring object made of various materials.

FIG. 5A illustrates light reflection of general print. The general print has low angle dependency. In this case, when the print is observed from various angles, change in color of the object does not occur relatively. This is because the print has an uneven surface, thereby resulting in diffusion of light emitted on such a surface.

FIG. 5B illustrates light reflection of metal or pearl. The metal or the pearl as illustrated in FIG. 5B has strong luster and its coated surface has large angle dependency for the following two reasons: (1) the surface of the metal or the pearl is smooth; and (2) the metal or the pearl is of bright material.

The reason (1) will be described below. When the surface of the object is smooth, light emitted on the surface is reflected most strongly under a regular reflection condition (an incident angle and a light receiving angle are the same). In contrast, an amount of reflecting light is decreased in a diffuse reflection direction (a direction other than the regular reflection). Hence, for example, in photographing the object, the image obtained under the highlight condition is very bright (in this case, the highlight condition is a condition for emitting light strongly). In contrast, the image obtained under the shade condition is dark quickly (in this case, the shade condition is a condition for emitting light weakly).

The reason (2) will be described below. A coating surface of the object processed with metal or pearl (e.g., a coating surface of a car) has aluminum flake called bright material, or material called mica flake. For this reason, brightness or color of the object greatly changes depending on an incident angle of light emitted on the coating surface. The reflection of the aluminum flake and the mica flake will be described below.

FIG. 5C illustrates light reflection of an aluminum flake. A coating surface of the measuring object having the aluminum flake has large angle dependency of light emitted within the coating layer of the surface. This is because the light is strongly reflected by aluminum metal of the coating layer.

FIG. 5D illustrates light reflection of a mica flake. A coating surface of the measuring object having the mica flake changes color according to a reflection angle of light. This is because the light is interfered by the mica flake.

In the present embodiment, since the photographing unit 21 is a color camera, color information on a given image can be obtained by the photographing unit 21. In such a way, the photographing unit 21 can obtain RGB data corresponding to colorful-glittering properties of, for example, a pearl coating surface made of the mica flake.

However, since the surface having metal, pearl, or the like has large angle dependency, color of such a surface may greatly change according to lighting angles with respect to the surface. For this reason, when a color conversion formula specific for a single angle for converting RGB into XYZ applies to other angles, a conversion error may be increased.

Accordingly, the present embodiment provides multiple color conversion formulas with respect to different lighting angles or light receiving angles to perform color conversion by use of the multiple color conversion formulas with respect to the respective angles whereby it is possible to decrease the conversion error.

In order to digitalize the texture, such as the above "glittering" with respect to an image of the measuring object, the following two points are critical.
(A) Visible color needs to be considered. Such color does not depend on devices. The color needs to correspond to the sensitivity of human color visual faculties. For example, such color needs to be represented by use of the tristimulus values XYZ or L*a*b* values.
(B) X, Y, and Z values in the (A) case need to be accurate. If the error of these values is increased, the error of the digitalized texture is increased. For this reason, when an RGB camera is used for photographing, a conversion error is needed to be as small to the extent possible.

Accordingly, the color conversion formula used in S8 of FIG.4 is preset so as to accurately convert the RGB luminance data into the tristimulus values XYZ. Specifically, the parameters a1 through a7 to be substituted into the conversion formula (For example, Formula 2 below) or conversion table for converting the RGB luminance data into the tristimulus values XYZ are preset. The conversion formula or conversion table corresponds to conditions for converting the captured image into the tristimulus values XYZ. The setting of such parameters (parameters for colorimetric conversion) will be described below.

SETTING OF PARAMETERS FOR COLORIMETRIC CONVERSION
FIG.6 is a diagram for explaining a state when parameters for colorimetry conversion are initialized upon shipping. Prior to shipping of the manufactured appearance characteristic measuring system 100, the following is prepared in advance for setting parameters for colorimetric conversion. With respect to multiple-plain-patches having different colors, a spectrometer 6 is used to obtain tristimulus values XYZ obtained by use of 15 degrees (highlight condition) and 45 degrees (shade condition), which correspond to the lighting angle conditions as described in FIG.1.

Further, the tristimulus values XYZ obtained by the spectrometer 6 are set as fixed true values. In this case, the parameters a1 through a7 to be used for a colorimetric conversion formula (see Formula 2 below) are set so as to match the fixed true values as much as possible.

Specifically, the tristimulus values XYZ obtained by the spectrometer 6 are preliminarily stored in a host system 9. In this case, for example, after the photographing unit 21 and the lighting units 11 and 12 are assembled prior to shipment, the tristimulus values XYZ obtained by the spectrometer 6 are stored as the fixed true values in the corresponding information processing apparatus 4 (or a controller 74 or storage 75 in FIG. 12) coupled to each appearance characteristic measuring system 100 (A, B and the like).

In setting the parameters for colorimetry conversion, the multiple-color-patches are used for setting in the corresponding appearance characteristic measuring system 100 (A or B) and spectrometer 6 of FIG. 6. In this example, as an example, the multiple-color-plain patches are patches called a CCS II color tile set with 12 color-ceramics-tiles.

The multiple-color-plain-patches may be 12 pieces of color tile in which reflectance is 100 % or less. However, it is preferable that the color tiles include a color tile whose reflectance is 100 % or more, or whose lightness is 100 or more. Since the reflection luminance of a coating surface having a bright material is relatively high, it is preferable to perform color calibration of these patches including a high-reflectance patch. Such calibration results in improvement of the color conversion accuracy in measuring the high-reflectance sample.

In order to optimize the color conversion formula, it is preferable to measure 8 or more colors by use of the color-plain-patches.

As an example, the instrument "BYK mac i" from BYK-Gardner is used as the spectrometer 6. The tristimulus values XYZ are calculated by use of the following: a spectral reflectance measured by the spectrometer 6, a color matching function corresponding to 10 degree field of view, and the light source 1.

Note that in FIG.6, as an example, the colorimetric values measured by the spectrometer 6 are stored via the host system 9 in the appearance characteristic measuring systems A and B. However, the colorimetric values measured by the spectrometer 6 may be directly transmitted to the appearance characteristic measuring systems A and B from the spectrometer 6 without the host system 9. In this case, the spectrometer 6 and the appearance characteristic measuring systems A and B are connected to each other.

FIG. 7 is an initial setting flowchart of the parameters for colorimetry conversion.

The flowchart of FIG.7 illustrates a process of pre-setting the parameters for the colorimetric value calculation of S8 in FIG.4. This setting process is performed prior to shipment of the manufactured appearance characteristic measuring system 100.

In S801, original data to be used by conversion formula is preliminarily stored in the storage (original data storage 846).

An example of the original data for colorimetry conversion is indicated by Formula 2 below. Formula 2 is a formula for converting RGB into XYZ. Note that, in Formula 2, a1 through a7 denote parameters.

As an example, the present embodiment illustrates Formula 2. However, a linear conversion formula or a cubic conversion formula may be used. When the RGB values are all zero, a constant term (e.g., a7 of Formula 2) may be set to zero such that XYZ values are zero.

In S802, the spectrometer 6 performs colorimetric measurement of the tristimulus values XYZ with respect to the 12-color-patches under the same angle condition as the highlight condition.

In S803, the spectrometer 6 performs colorimetric measurement of the tristimulus values XYZ with respect to the 12-color-patches under the same angle condition as the shade condition.

In S804, the tristimulus values XYZ obtained in S802 and S803 are, as fixed true values, stored in the fixed true value storage 847.

In S805, the photographing unit 21 of the appearance characteristic measuring system 100 obtains the RGB image (Raw data) with respect to the 12-color-patches by use of the lighting angle of the highlight condition.

In S806, the highlight RGB luminance data is created based on the Raw data obtained in S805.

In this case, a central image area of the Raw data is identified and further, an average value for each of R, G, and B within the identified image area is calculated. These average values correspond to the highlight RGB luminance data. A measuring area of the image captured by the photographing unit (camera) 21 is relatively larger in comparison to an area to be measured by the spectrometer 6. This is because deviation may occur in an image area other than the central image area. In consideration to the above point, in this example, the size of the central image area is 128 by 128 pixels for obtaining the highlight RGB luminance data. Note that before the average values calculated, noise processing, calibration processing or demosaic processing, as explained in FIG. 4, may be performed.

In S807, the photographing unit 21 of the appearance characteristic measuring system 100 obtains the RGB image (Raw data) with respect to the 12-color-patches by use of the lighting angle of the shade condition.

In S808, the shade RGB luminance data is created based on the Raw data obtained in S807. In this case, in the same way as S806, the central image area of the RGB data (Raw data) is identified and further, an average value for each of R, G, and B within the identified image area is calculated. These average values correspond to the shade RGB luminance data.

In S809, the parameter updating unit 85 reads out the conversion formula (Formula 2) stored in S801, and the fixed true values stored in S804.

In S810, the colorimetric conversion parameter updating unit 85 sets the parameters a1 through a7 from the highlight RGB luminance data created in S806, such that the tristimulus values XYZ converted by use of the conversion formula are close to the fixed true values XYZ defined under the highlight condition in S804.

In this case, the parameters a1 through a7 of the conversion formula (Formula 2) are calculated based on the highlight RGB luminance data (See S806) with respect to the aforementioned 12-colors, and the tristimulus values XYZ (see S802) with respect to the 12 colors measured under the highlight condition.

For example, the parameters a1 through a7 are calculated by use of a least-squares method. In this case, X, Y, and Z values obtained under the highlight condition with respect to the 12-color-patches measured by the spectrometer 6 are set to criterion variables. Further, R, G, and B values obtained under the highlight condition with respect to the 12-color-patches measured by the photographing unit 21 are set to explanatory variables.

The conversion formula into which the parameters a1 through a7 are substituted is, as the highlight calculation conversion formula, stored in the highlight conversion formula storage 843.

In S811, the colorimetric conversion parameter updating unit 85 sets parameters from the shade RGB luminance data created in S808, such that the tristimulus values XYZ converted by use of the shade RGB luminance data are close to the fixed true values XYZ defined under the shade condition in S804.

In this case, the parameters a1 through a7 of the color conversion formula (Formula 2) are calculated based on the shade RGB luminance data (see S808) with respect to the aforementioned 12 colors, and the XYZ data (see S803) with respect to the 12 colors measured by the spectrometer 6 under the shade condition. In the same way as S810, for example, the parameters a1 through a7 are calculated by use of the least-squares method, where the X, Y, and Z values obtained under the shade condition with respect to the 12-color-patches are set to criterion variables. Further, the R, G, and B values obtained under the shade condition with respect to the 12-color-patches are set to explanatory variables.

The conversion formula into which the parameters a1 through a7 are substituted is, as the shade calculation conversion formula, stored in the shade conversion formula storage 844.

As explained above, the color conversion formulas corresponding to each of the lighting angles (the highlight condition and the shade condition) are determined by use of the XYZ data (tristimulus values XYZ) obtained by the spectrometer 6, and the RGB data obtained by the photographing unit 21. In such a way, the photographed RGB data can be converted into the XYZ data by use of the conversion formulas in the appearance characteristic measuring system 100.

Accordingly, the appearance characteristic measuring system 100 of the present embodiment can measure the XYZ data per pixel with respect to a given image having a wider measuring area than a measuring area to be measured by the spectrometer 6. In this case, the XYZ data is accurately calculated so as to be close to values of the image measured by the spectrometer 6.

Note that in the present embodiment, as an example, the 12-color-tiles are used as the multiple-color-plain-patches for creating the color conversion formulas. However, for example, patches having more than one hundred colors may be used for converting RGB into XYZ.

FIG. 8 is a flowchart illustrating an example of updating the parameters for colorimetry conversion.

As explained above, the parameters a1 through a7 for colorimetry conversion are initially set by use of the colorimetry conversion formula (Formula 2). These parameters are accurately calculated so as to be close to values of the image measured by the spectrometer 6. However, these parameters are preferably changed as needed for the following reasons. The light source 1 may be deteriorated over time, or the fixed true values may be shifted when the lighting units 11 and 12 or a lens of the photographing unit 21 is replaced.

In "Start" block of FIG. 8, the updating process of FIG. 8 is implemented when an LED of the light source 1 is deteriorated over time, or alternatively, the fixed true values are shifted when each of the lighting units 11 and 12 or a lens of the photographing unit 21 is replaced. In FIG. 7, the fixed true values as well as the parameters a1 through a7 are set. In FIG. 8 the parameters a1 through a7 are updated by use of the multiple-color-plain-patches in the appearance characteristic measuring system 100. The fixed true values are not updated.

The processes of S901 through S907 of FIG.8 are approximately similar to the processes of S805 through S811 of FIG.7.

In S906, the parameters a1 through a7 are updated such that, under the highlight condition, the tristimulus values XYZ converted by use of the conversion formula (Formula 2) from the presently measured highlight RGB luminance data (see S806 of FIG. 7), are close to the corresponding fixed true values predefined in S804. The conversion formula (Formula 2) into which the updated parameters a1 through a7 are substituted is stored in the highlight conversion formula storage 843. Such process allows the highlight conversion formula to be updated.

In S907, the parameters a1 through a7 are updated such that, under the shade condition, the tristimulus values XYZ converted by use of the conversion formula (Formula 2) from the presently measured shade luminance RGB data (see S808 of FIG. 7) are close to the corresponding fixed true values predefined in S804. The conversion formula (Formula 2) into which the updated parameters a1 through a7 are substituted is stored in the shade conversion formula storage 844. Such process allows the shade conversion formula to be updated.

In such a way, the parameters a1 through a7 to be used in the colorimetric conversion formula or the conversion table are updated under the respective setting conditions whereby the colorimetric values (i.e., tristimulus values XYZ obtained by use of the colorimetric conversion formula (Formula 2)) are accurately set at any time so as to be close to the true values measured by the spectrometer 6.

VERIFICATION OF COLORIMETRIC VALUES
Next, explanation will be provided for accuracy of the colorimetric values (i.e., values XYZ obtained by use of Formula 2).

In order to check if the above colorimetric values are accurate, error in the following two cases is compared.
(i) A first case that the shade RGB luminance values (see S808 of FIG. 7) obtained under the shade condition are converted into the tristimulus values XYZ (see S811 of FIG. 7) by use of the shade color conversion formula (Formula 2).
(ii) A second case that, without the conversion formulas with respect to various angles, the RGB values photographed under the shade condition are converted into the tristimulus values XYZ by use of the shade color conversion formula.

FIG. 9 illustrates the result of the tristimulus values XYZ obtained in the first case.

In FIG.9, Δx, Δy, and Δz denote absolute values of differences between X, Y and Z obtained by a commercially available spectrometer (e.g., instrument called "BYK mac i" from the BYK Gardner supplier). X, Y, and Z obtained in the first case (in this case, RGB data photographed by the photographing unit 21 is converted into XYZ data). The absolute values of FIG.9 are errors.

Note that the size of the image data photographed by the photographing unit 21 is 128 by 128 pixels. The values XYZ of the image data indicate average values. In FIG.9, an average error about X is 0.7, an average error about Y is 0.46 and an average error about Z is 0.49. These errors are relatively small.

FIG. 10 illustrates the result of the tristimulus values XYZ obtained in the second case (in this case, the RGB values photographed under the shade condition were converted into XYZ values). In FIG.10, an average error about X is 12.63, an average error about Y is 9.72 and an average error about Z is 20.93. These errors are relatively large.

From the results of FIG. 9 and FIG. 10, it has been found that conversion accuracy is improved by converting RGB data into XYZ data by use of the conversion formulas with respect to various lighting angles, as discussed in the first case.

In other words, the present embodiment can reduce a conversion error, such as an error which may be increased if the color conversion table (Look-up table) specific for a certain angle is applied to other angles in converting the RGB data into the XYZ data.

In the present embodiment, the photographed RGB data is used as Raw data for converting the RGB data into the XYZ data. However, the photographed RGB data may not be used as Raw data and be normalized based on the white data measured by use of a white-reference-plate, etc. In this case, the photographed RGB data per pixel is normalized based on the white data and the normalized RGB data may be used for converting. This enables corrections to time-related illumination unevenness with respect to the lighting unit.

SECOND PROCESS OF CALCULATING COLORIMETRIC VALUES
In the process of calculating colorimetric values of FIG. 4, data to be processed is obtained with a single shot with respect to the respective angles, but may be obtained in a different manner. In general, industrial products have different lightness, such as a high level or a low level. For this reason, when a sample having high level lightness is photographed with a constant exposure time of a camera, a dynamic range of the camera may be over a maximum. On the other hand, when a sample having a low level lightness is photographed, the photographed data may not be clearly visible.

In view of the above situation, in this second process, a sample is photographed with different exposure times, i.e., a multi-exposure, for obtaining RGB data. Further, the obtained RGB data is combined and the combined RGB data is then color-converted. Such method allows samples having a lightness range from a high level to a low level to be measured accurately.

Explanation will be provided below for combination of RGB data according to this process. In the following description, for easy explanation, the combination processed under the highlight condition will be described. The combining process implemented under the shade condition is similar to the combining process implemented under the highlight condition.

In this case, exposure times have three types, e.g., "1/2T seconds", "T seconds", and "2T seconds". Where T denotes a reference exposure time, 1/2T denotes one half of T, and 2T denotes doubled T.

FIG. 11 is a detailed flowchart illustrating an example of combined RGB data according to a second control example. Note that, as an example, FIG. 11 illustrates a case that, when measured, the sample S is photographed and the colorimetric values are measured by use of the photographed data. The processes of updating the parameter by use of a specific-color-patch are the same as FIG. 11.

In S101, it is checked that the sample S is disposed on the inspection table 3.

In S102, the first lighting unit 101 is lighted.

In S103, RGB luminance data with respect to the sample S is obtained with the exposure time of 1/2T, T, and 2T by use of the lighting angle of the highlight condition. The obtained RGB luminance data is stored in the highlight image storage 831.

Specifically, the camera of the photographing unit 21 photographs the sample S three times with the exposure time of 1/2T, T, and 2T. The image data photographed with 1/2T is the darkest, while the image data photographed with 2T is the brightest. With respect to the photographed image data, in the same way as S806, the central image area of the corresponding image data is identified and further, an average value (i.e., RGB luminance data) for each of R, G, and B within the identified image area is calculated. In such a way, three patterns of highlight RGB luminance data (in this case, e.g., highlight RBG luminance data obtained with 1/2T, T, and 2T) are created.

In S104 through S109, the process of combining RGB data is implemented. In this description, data obtained with 2T is referred to as a first data. Data obtained with T is referred to as a second data. Data obtained with 1/2T is referred to as a third data.

In S104, the combination RGB data creating unit 863 identifies a pixel location (first saturated pixel area) in which a value of R of the RGB data (all pixel data referred to as the first data) obtained with the exposure time 2T is saturated or is close to a saturation value. In this case, since a 10-bit camera is used as the photographing unit 21, the saturation value is 2 to the power of ten minus 1, i.e., 1023. For example, when the luminance value of the first data is 1000 or more, the pixel location having such a value is identified as the first saturated pixel area.

In S105, the combination RGB data creating unit 863 replaces the luminance value of the pixel location identified in S104 with the luminance value of the pixel data (second data) obtained with the exposure time T. That is, the saturated pixel area is replaced with the pixel area in which a value is not saturated.

In S106, the combination RGB data creating unit 863 reduces by 1/2 the luminance value of the pixel location of the image data (first data) that is not identified in S104. That is, the value of the pixel area that is not replaced in S104 is reduced by 1/2 in order to normalize such a value based on the exposure time T.

In S107, the combination RGB data creating unit 863 identifies a pixel location in which a luminance value of R of the RGB data (pixel data referred to as the second data) obtained with the exposure time T, as replaced in S105, is saturated or is close to a saturation value. For example, the pixel location having the luminance value of 1000 or more is identified.

In S108, the combination RGB data creating unit 863 replaces the luminance value of the pixel location identified in S107 with the luminance value of the pixel data (third data) obtained with the exposure time 1/2T.

In S109, the combination RGB data creating unit 863 doubles the luminance value of the pixel area of the image data (third data) replaced in S108. In a such way, the exposure time T is taken as reference for normalization.

Next, flow proceeds to S8 and S9 of FIG. 4, and the combined RGB data created in FIG. 11 is substituted into the conversion formula for obtaining the XYZ data and the texture is then digitalized.

Note that FIG. 11 illustrates the case that the replaced luminance value is doubled or reduced by 1/2 for normalizing. However, data obtained with the exposure time of 2T and 1/2T is preliminarily normalized and these normalized data may be used for replacing.

In this example, when the luminance value of only the R data is saturated, the luminance value of the R data is replaced (see S105 or S108). Alternatively, the G and B data in the same location as the R data to be changed may also be replaced. The process to be employed can be determined in consideration of how easy it is to create the intended program (algorithm). Further, a procedure that is substantially the same as the one illustrated in FIG. 11 may additionally be performed with respect to each of the G data and the B data.

As explained above, the RGB luminance data obtained with multi-exposure are preliminarily created in order to properly implement the process of FIG. 11.

For this reason, in obtaining the XYZ data obtained by the spectrometer 6 (See FIGs 7 and 8), the XYZ data is obtained with the multiple exposure conditions under the highlight condition and the shade condition.

By use of the combined RGB data with respect to 12-color-plain-patches as obtained in such a way, the parameters a1 through a7 to be substituted into the colorimetric value conversion formula (Formula 2) is calculated to determine the conversion formula (Formula 2), as illustrated in FIG. 7 and FIG. 8.

In the present embodiment, combining the data obtained with multi-exposures increases dynamic ranges of the data in a pseudo manner, and thereby generating HDR (high Dynamic Range) data. Hence, various samples having very high luster and high brightness, and a sample with low luster and low brightness can be measured in such a state that data does not have crushed shadows and saturation values.

According to the appearance characteristic measuring system 100, the color conversion formulas or the color conversion tables are created by use of the HDR data with respect to the multiple-color-patches. Further, the sample S is measured by use of the HDR data. Accordingly, it is possible to improve colorimetric-value-conversion accuracy and texture-digitalization accuracy.

INTEGRATED MEASURING APPARATUS
FIG. 12 is a schematic diagram illustrating a configuration example of an integrated appearance characteristic measuring apparatus.

With reference to FIG. 1, as the example, explanation has been provided above for the case that, in the appearance characteristic measuring system 100, the information processing apparatus 4 is separate from the lighting units 11 and 12 and the photographing unit 21. However, the controller and the storages configured to implement the information processing function may be integrated with the lighting units 11 and 12 and the photographing unit 21.

In FIG. 12, an integrated appearance characteristic measuring apparatus 7 as a whole is covered with a housing 70. The integrated appearance characteristic measuring apparatus 7 includes a lighting unit 71, a lighting unit 72, a photographing unit (camera) 73, a controller 74, a storage 75, a display 76, operation buttons 77, and the like. The integrated appearance characteristic measuring apparatus 7 is an example of the measuring apparatus according to the present embodiment.

This integrated appearance characteristic measuring apparatus 7 conducts photographing in a state that the housing 70 is in contact with the surface of the sample S.

Note that the housing 70 has holes through which light from lighting units 71 and 72 is emitted. The photographing unit 73 can photograph an image reflected on the sample 3 through the hole.

In this example, the first lighting unit 71 irradiates the sample S with light so as to be reflected on the sample S in a vicinity of the regular reflection direction with respect to the photographing unit 73 (highlight condition). Further, the second lighting unit 72 irradiates the sample S with light so as to be reflected on the sample S in the diffusion direction with respect to the photographing unit 73 (shade condition).

The controller 74 and the storage unit 75 perform functions of the color measurement of the information processing apparatus 4 of FIG. 3.

The display 76 is a color display, such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence), or a monochrome display. The display 76 has the function of the monitor 5 and can display a setting menu and an operation menu in addition to image data, operating results, and the like.

The operation buttons 77 are input devices for giving instructions from an operator.

In FIG. 12, the lighting units 71 and 72 and the photographing unit 73 are provided at an upper portion of the integrated appearance characteristic measuring apparatus 7. The sample S is disposed at the lower portion of the integrated appearance characteristic measuring apparatus 7. However, for example, the integrated appearance characteristic measuring apparatus 7 may be disposed horizontally so as to dispose the sample S at a wall portion as long as the housing 70 can be in contact with the sample S.

With this configuration, the respective conversion formulas or conversion tables are created by use of the respective lighting angle conditions whereby it is possible to improve conversion accuracy in converting the photographed RGB data into the XYZ data.

With reference to FIG. 12, as the example, explanation has been provided above for the case that the number of lighting units is 2 and the number of photographing units is one. However, as illustrated in FIG. 13 below, the integrated appearance characteristic measuring apparatus 7 may include two cameras and one lighting unit.

SECOND EMBODIMENT
Next, explanation will be provided for the appearance characteristic measuring apparatus of the present embodiment. FIG. 13 is a diagram illustrating an example of an appearance characteristics measuring system 100A according to the second embodiment.

In the first embodiment, explanation has been provided above for the case that the multiple lighting units are provided for measuring images by use of the multiple lighting angles. However, as illustrated in FIG.13, multiple photographing units (cameras 23 and 24) may be provided for photographing images by use of multiple photographing angles.

In FIG. 13, a first photographing unit (camera) 23 and a second photographing unit (camera) 24 are disposed so as to conduct photographing at different angles with respect to the sample S. Hence, a photographing device 2A having two photographing units 23 and 24 can photograph the sample S on the inspection table 3 by use of two photographing angles. Note that the number of photographing angles may be increased by increasing the number of cameras.

In the present embodiment, an instruction for selecting the lighting unit is not given, but an instruction for selecting the photographing unit (camera 23 or camera 24) to be used for photographing is given. For this reason, in the information processing apparatus 4 of the present embodiment, a photographing controller 82 selects the photographing unit 23 or the photographing unit 24, instead of the lighting controller 81 of FIG. 3 for selecting the lighting units 11 or the lighting unit 12.

In the present embodiment, the multiple photographing angles are used for photographing whereby it is possible to obtain measuring values corresponding to the sensitivity of vision when the sample S is observed.

In the present embodiment, the first photographing unit 23 photographs the sample S irradiated with light from the lighting unit 14. In this case, the light is reflected on the sample S in a vicinity of the regular reflection direction with respect to the lighting unit 14 (highlight condition). Further, the second photographing unit 24 photographs the sample S irradiated with light from the lighting unit 14. In this case, the light is reflected on the sample S in the diffusion direction with respect to the lighting unit 14 (shade condition).

In this case, instead of the lighting angle conditions, the photographing angle conditions are used for creating the conversion formulas or conversion tables whereby it is thus possible to improve conversion accuracy in converting the photographed RGB data into the XYZ data.

MODIFICATION
With reference to FIG. 13, explanation has been provided above for the case that the multiple photographing units 23 and 24 obtain the images by use of the different photographing angles. However, one photographing unit (photographing device), such as a line sensor 20 as illustrated in FIG. 14, may obtain the images by use of the different photographing angles.

The line sensor 20 of FIG. 14 is a line-scan-type photographing device. The line sensor 20 can obtain a single image obtained with use of multiple photographing angles, by changing the light angle or photographing angle continuously.

The line sensor 20 photographs (images) the sample S whereby it is possible to obtain the image data at once by use of at least the multiple light angles or the multiple photographing angles.

THIRD EMBODIMENT
With reference to FIG. 15, explanation will be provided below for a case of the third embodiment in which a spectroscopic camera 25 is used as the photographing unit.

The spectroscopic camera 25 of the present embodiment can obtain two-dimensional spectroscopy information on a wavelength band with respect to a visible light region. For example, the spectroscopic camera 25 may be a multi-spectrum camera for obtaining spectroscopy information on a multiple wavelength bands, or a hyper spectrum camera for obtaining spectroscopy information with high-wavelength resolving power.

Next, with reference to FIG. 15, explanation will be provided below for a schematic process of dispersing light to be obtained by the spectroscopic camera (photographing unit) 25. FIG. 15 is a diagram illustrating an example of a main configuration of a photographing unit (spectroscopic camera) 25 of an appearance characteristic measuring system according to the third embodiment. As an example, the spectroscopic camera 25 includes a set of filters and a multi-spectrum camera having a diffraction grating. However, the spectroscopic camera 25 may be a hyperspectral camera including one or more sets of filters and a diffraction grating (or a prism), or the like.

The spectroscopic camera 25, which is implemented as a spectral information obtaining unit, can obtain the spectral information per microlens according to the number of spectral filters, by use of a group of spectral filters 56a, 56b and 56c of a main lens 54, and a microlens array 53 disposed between the main lens 54 and a receiving light element. The spectral information obtaining unit obtains the two-dimensional spectral information.

As illustrated in FIG. 15, the microlens array (MLA) 53 having multiple microlenses (small lens) is disposed in a vicinity of a condensing position of the main lens 54. A receiving light element array 55, which has multiple receiving light elements (sensors) for converting optical information condensed by the main lens 54 into electronic information (electrical signal), is disposed on a light receiving plane.

In FIG. 15, for ease of understanding in the following description, the main lens 54 as an optical system is a single lens and further, a diaphragm position S of the main lens 54 is indicated as a center of the single lens. However, as illustrated in FIG. 15, a color filter 56 is actually not disposed within the lens 54, but is disposed in a vicinity of the diaphragm position. The "vicinity of the diaphragm position" includes the diaphragm position and means a portion through which light with respect to various angles of view can pass. In other words, the "vicinity of the diaphragm position" means a range of allowed design positions of the color filter 56 with respect to the main lens 54.

In FIG. 15, the color filter 56 as an optical bandpass filter is disposed in a center of the main lens 54. The color filter 56 is a filter that corresponds to the tristimulus values XYZ and that has spectral transmittances based on a color matching function in the XYZ colorimetric system. In this case, the color filter 56 has multiple (e.g., 3) filters 56a, 56b and 56c, which have different spectral transmittances.

The above optical bandpass filter may be a combination of multiple filters having different spectral transmittances, or be a single filter having different spectral transmittances for each filter portion.

In such a configuration, for example, with respect to wavelength bands from 380nm through 780nm, when 31-types of optical bandpass filters have transmission wavelength peaks, each in 10nm, these filters can obtain the spectral information on the peak of each 10nm.

As explained above, a range of spectral wavelengths measured (photographed) by the camera covers from 380nm to 780nm in a visible light region. Accordingly, it is possible to obtain the spectral information corresponding to the sensitivity of human vision.

In the present embodiment, the conversion formulas or the conversion tables are also preliminarily set by use of the different angles, as explained in the aforementioned embodiments, and thereby improving the conversion accuracy. Using 4 or more channels expands information amount and thus enables higher conversion accuracy than using 3 channels.

Explanation has been provided above for the embodiment employing the RGB camera and the multi-spectral camera. However, the XYZ camera may be used for performing the same process as the process described above. Even if a filter having characteristics similar to characteristics of the color matching function (e.g., Formula 2), or the like is used as the camera, the characteristics of the color data obtained by use of the filter may not match the characteristics of the color matching function. In this case, as a result, a conversion error may occur. In view of the above point, as an example, in the present embodiment, the RGB data may be implemented as pseudo data of the XYZ camera for creating the color conversion formula in the same manner as the aforementioned embodiments.

The camera of the photographing unit is not limited to a single plate camera. For example, a three-plate camera may be used as the camera of the photographing unit.

Explanation has been provided above for the case that the lighting units are white-color lighting units, and the photographing unit is a color camera. However, the lighting units may be LEDs for emitting light with R, G and B colors. The photographing unit may be a monochrome camera. In this case, the LED sequentially emits light for each of R, G, and B colors and further, the monochrome camera conducts photographing three times accordingly. Such a configuration can obtain the RGB data. Advantageously, demosaic processing is not required. On the other hand, photographing is required three times with respect to the same photographing angle whereby measuring time may be increased. Any aspect can be applied depending on the situation.

Explanation has been provided above for preferred embodiments, but is not intended to limit to a specific embodiment. Various modifications or changes can be made within the scope of the present disclosure.

100, 100A, 100B　　appearance characteristic measuring system (measuring device)
1　　light source
2　　photographing device
3　　inspection table
4A, 4B, 4C　　information processing apparatus
5　　monitor
6　　spectrometer (colorimeter)
7　　integrated appearance characteristic measuring apparatus (measuring device)
9　　host system
11　　lighting unit (first lighting unit, LED)
12　　lighting unit (second lighting unit, LED)
14　　lighting unit
2, 2A, 2C 　　photographing device
20　　multi-spectrum camera
21　　photographing unit (camera)
23　　first photographing unit (camera)
24　　second photographing unit (camera)
25　　spectroscopic camera
71　　first lighting unit
72　　second lighting unit
73　　photographing unit
74　　controller
75　　storage
76　　display
77　　operation buttom
80A　　input
80B　　monitor output
83　　image data storage
84　　calculation data storage (storage)
85　　colorimetric conversion parameter updating unit
86　　colorimetric value calculator(converter)
863　　combination RGB data creating unit (RGB data combining unit)
864　　tristimulus values XYZ calculator
87　　texture calculator
88　　measuring data storage
S　　sample (measuring object)
P　　color tile (multiple-color patches)

The present application is based on and claims priority to Japanese Patent Application No. 2018-37927 filed on March 2, 2018, the entire contents of which are hereby incorporated herein by reference.

Claims

A measuring device for measuring an object to be measured, the measuring device comprising:
at least one lighting unit configured to irradiate the object with light;
at least one photographing unit configured to photograph the object irradiated with the light to produce a captured image; and
a converter configured to convert the captured image into tristimulus values,
wherein the photographing unit is configured to conduct photographing multiple times by use of multiple setting conditions for changing at least one of a lighting angle of the lighting unit and a photographing angle of the photographing unit, and
　　　　in the converter, a condition for converting the captured image into the tristimulus values being different for each of the setting conditions.
　　　　The measurement device according to claim 1, further comprising a storage, wherein the lighting unit and the photographing unit are disposed so as to meet a first setting condition for irradiating the object with light from a first angle with respect to the photographing unit, and a second setting condition for irradiating the object with light from a second angle with respect to the photographing unit, the second angle is different from the first angle, and
the storage is configured to store, for each of the first setting condition and the second setting condition, a setting-condition-specific conversion condition under which data in each pixel of the captured image is converted into the tristimulus values.
The measuring device according to claim 2, further comprising a parameter setting unit, wherein the storage is configured to store:
　　　　　　　　predetermined true color values for reference with respect to multiple-color-plain patches;
　　　　　　　　a conversion formula or a conversion table used by the converter for converting the captured image into the tristimulus values; and
　　　　　　　　updatable parameters substituted into the conversion formula or the conversion table, and
wherein the parameter setting unit is configured to set or update the parameters to be substituted into the conversion formula or the conversion table, such that tristimulus values converted from a photographed image are close to the true color values for each of the first setting condition and the second setting condition, the photographed image is obtained by photographing the multiple-color-plain patches under each of the first setting condition and the second setting condition.
The measuring device according to claim 2 or 3, wherein the first setting condition is a highlight condition for irradiating the object with light from a highlight angle with respect to the photographing unit, the highlight angle allows the light to be reflected on the object in a regular reflection direction or a vicinity direction of the regular reflection direction to enter the photographing unit, and
the second setting condition is a shade condition for irradiating the object with light from a shade angle with respect to the photographing unit, the shade angle allows the light to be reflected on the object in a diffuse reflection angle to enter the photographing unit.
The measuring device according to any one of claims 1 through 4, wherein the at least one lighting unit is an LED for which a color rendering evaluation index Ra is 95 or more, or an LED for emitting light with 3 or more colors.
　　　　　The measuring device according to any one of claims 1 through 5, wherein data photographed by the photographing unit is 3-channel color data.
　　　　　The measuring device according to claim 6, wherein the photographing unit is configured to conduct photographing with multiple exposure times under each of the multiple setting conditions to obtain RGB data,
　　　　　wherein the converter includes an RGB data combining unit configured to combine the RGB data obtained with the multiple exposure times, and
　　　　　wherein the RGB data combining unit is configured to obtain RGB combined data by performing, for each of R, G, and B colors, a procedure specific to each corresponding color, the procedure including:
identifying a first saturated pixel area in which a value of the corresponding color of the RGB data obtained with a longest exposure time is saturated or is close to a saturation value, and replacing the RGB data in the first saturated pixel area with the RGB data obtained with the second longest exposure time;
decreasing a luminance value obtained with the longest exposure time based on a reference exposure time when the longest exposure time is longer than the reference exposure time;
identifying a second saturated pixel area within the first saturated pixel area when the RGB data obtained with the second longest exposure time in the second saturated pixel area has a saturated value or value close to a saturation value, and replacing the RGB data in the second saturated pixel area with the RGB data obtained with a third longest exposure time; and
increasing a luminance value obtained with the third exposure time based on the reference exposure time when the third longest exposure time is shorter than the reference exposure time.
The measuring device according to any one of claims 1 through 7, further comprising a texture calculator configured to digitalize, as texture, appearance characteristics of the object by use of the converted tristimulus values.
The measuring device according to claim 8, wherein the texture calculator includes:
an L*a*b* calculator configured to convert XYZ indicating the tristimulus values per pixel into L*a*b* values per pixel in an L*a*b* color system, and
a texture evaluating unit configured to evaluate the texture based on a variation amount corresponding to dispersion values in an image with the L*a*b* values.
The measuring device according to any one of claims 1 through 5, 8 and 9, wherein data photographed by the photographing unit is pseudo XYZ data, or data with 3 or more channels photographed by a multi-spectrum camera.
A method for setting a color measuring conversion parameter in a measuring device,
wherein the measuring device comprises a storage, at least one lighting unit, and at least one photographing unit, the lighting unit and the photographing unit are disposed so as to meet a first setting condition for irradiating multiple-color-plain patches with light from a first angle with respect to the photographing unit, and a second setting condition for irradiating the multiple-color-plain patches with light from a second angle with respect to the photographing unit, the second angle is different from the first angle,
the method comprising;
obtaining, in the storage, a conversion formula or a conversion table for colorimetric conversion, the conversion formula or the conversion table storing updatable parameters;
obtaining, in the storage, fixed true color values for reference with respect to the multiple-color-plain patches, for each of the first setting condition and the second setting condition;
photographing, by the measuring device, the respective multiple-color-plain patches under the first setting condition to obtain a first photographed image;
photographing, by the measuring device, the respective multiple-color-plain patches under the second condition to obtain a second photographed image;
setting or updating first parameters to be substituted into the conversion formula or the conversion table, such that tristimulus values converted from the first photographed image are close to the true color values obtained under the first setting condition; and
setting or updating second parameters to be substituted into the conversion formula or the conversion table, such that tristimulus values converted from the second photographed image are close to the true color values obtained under the second setting condition.
The method according to claim 11, wherein the obtaining, in the storage, fixed true color values for reference includes:
performing colorimetry, by a colorimeter, to obtain first tristimulus values by use of the first angle of the first setting condition with respect to the multiple-color-plain patches;
performing colorimetry, by the colorimeter, to obtain second tristimulus values by use of the second angle of the second setting condition with respect to the multiple-color-plain patches; and
storing, as the color fixed true values for reference, the first and second tristimulus values in the storage.
The method according to claim 11 or 12, wherein the multiple color-plain patches include a patch whose reflectance is 100 or more, and whose lightness is 100 or more.
The method according to any one of claims 11 through 13, wherein the multiple color-plain patches include at least 8 colors.
An industrial product manufactured based on inspection performed by the measuring device according to any one of claims 1 through 10.